News for hackers. Stuff that matters.
Re:PEP 686 – Make UTF-8 mode default by rootext on Friday April 26, 2024 @10:00AM (#40169499)
It seems you are mixing two things: inner string representation and read/write encoding. Java has never used UTF-16 as default for the second.
Re:PEP 686 – Make UTF-8 mode default by cryptonector on Friday April 26, 2024 @11:39AM (#40170808)
Not even on Windows?
Re:PEP 686 – Make UTF-8 mode default by layer8 on Friday April 26, 2024 @12:16PM (#40171134)
No, file I/O on Windows in general doesn’t use UTF-16, but the regional code page, or nowadays UTF-8 if the application decides so.
Re:PEP 686 – Make UTF-8 mode default by int_19h on Friday April 26, 2024 @05:34PM (#40174570)
Depends on what you define as "file I/O", though. NTFS filenames are UTF-16 (or rather UCS2). As far as file contents, there isn't really a standard, but FWIW for a long time most Windows apps - Notepad being the canonical example when asked to save anything as "Unicode" would save it as UTF-16.
Re:PEP 686 – Make UTF-8 mode default by layer8 on Friday April 26, 2024 @06:21PM (#40174958)
I'm talking about the default behavior of Microsoft's C runtime (MSVCRT.DLL) that everyone is/was using.

UTF-16 text files are rather rare, as is using Notepad's UTF-16 options. The only semi-common use I know of is *.reg files saved from regedit. One issue with UTF-16 is that it has two different serializations (BE and LE), and hence generally requires a BOM to disambiguate.

Re:PEP 686 – Make UTF-8 mode default by TheCycoONE on Friday April 26, 2024 @10:43PM (#40176676)
Powershell use to output utf-16 by default on Windows. It might still but it's been awhile since I needed to try.
Re:PEP 686 – Make UTF-8 mode default by int_19h on Saturday April 27, 2024 @06:27AM (#40178858)
Then you're talking about the C stdlib, which, yeah, is meant to use the locale-specific encoding on any platform, so it's not really a Windows thing specifically. But even then someone could use the CRT but call wfopen() rather than fopen() etc - this was actually not uncommon for Windows software precisely because it let you handle Unicode without having to work with Win32 API directly.

Microsoft's implementation of fopen() also supports "ccs=..." to open Unicode text files in Unicode, and interestingly "ccs=UNICODE" will get you UTF-16LE, not UTF-8 (but you can do "ccs=UTF-8"). .NET also has this weird naming quirk where Encoding.Unicode is UTF-16, although there at least UTF-8 is the default for all text I/O classes like StreamReader if you don't specify the encoding. Still, many people didn't know better, and so some early .NET software would use UTF-16 for text I/O for no reason other than its developers believing that Encoding.Unicode is obviously what they are supposed to be using to "support Unicode", and so explicitly passing it everywhere.

Re:PEP 686 – Make UTF-8 mode default by Dwedit on Friday April 26, 2024 @10:36PM (#40176650)
Or possibly confusing it with JavaScript, which treats strings as sequences of UTF-16 characters?