Post

🏆 Top 5 Python 3.15 features to start using today: 1. Lazy imports 2. New built-in `frozendict` 3. New built-in `sentinel` 4. Unpacking inside comprehensions 5. New sampling profiler Tachyon What's your top Python 3.15 new feature?

I think my top 5 are: 1. UTF-8 as the default encoding on Windows 2. The new sentinel type 3. Unpacking in comprehensions: [*sub for sub in nested] 4. The new sampling profiler 5. pprint's new expand=True But it's hard to choose!

What's up with the Windows UTF-8 default? Why is it such a big thing? I haven't used Windows in a while, so maybe I forgot.

Basically Windows has been reading/writing with an encoding of cp1252 by default for forever. Both it and utf-8 are ASCII supersets but DIFFERENT ones. Valid text in cp1252 can result in errors or emojibake in utf-8 and vice versa. It's a cross-platform mess. And once 3.14 is EOL'd it'll be done.

How can I check for myself that the default encoding on Windows is cp1252 _without_ a Windows machine? Can you help me?

IIRC, look at the docs for open and the notes on the default value for encoding. It mentions some module’s function like sys.getdefaultencoding (making this up!) or some such.

Yeah, that's what I did. I went down the rabbit hole for a while but I never got to a place that said “and on Windows that's cp1252”, or any other encoding for that matter

…suspect that you’d have to go down the windows docs rabbit hole. I’m positive that the default encoding will vary depending on the locale: maybe cp1252 for western places like Europe, but likely others in China, Japan, Saudi Arabia, Armenia… (isn’t text wonderful?)

This ☝️ It uses a "code page" which is a locale-specific encoding. So cp1252 will only be the default on machines set to English or similar character sets. I don't think the Python docs list all the different encodings. You'd probably need to dive through MS docs for that.