Re:Microsoft is not rebasing Windows to Linux
by shadowwolf007 on Monday October 12, 2020 @01:00PM
(#24756564)
As of Windows 10 1903 you can set the codepage of the application to UTF-8 - in fact for UWP Microsoft recommends you do that (https://docs.microsoft.com/en-us/windows/uwp/design/globaliz...).
Unfortunately I don't see that being common enough for data ingestion tools for this to be relevant for a very, very long time.
For now, it's just another pain point because you can't assume Windows == UTF-16 lookalike anymore.
Re:Microsoft is not rebasing Windows to Linux
by fomine3 on Tuesday October 13, 2020 @12:11AM
(#24762204)
Is it now recommended rather than experimental? I expect some legacy apps that uses local OEM codepage would be broken.
Re:Microsoft is not rebasing Windows to Linux
by user5994461 on Monday October 12, 2020 @09:43AM
(#24754245)
Windows is UTF-16 internally while Linux is UTF-8.
If Windows goes to unicode by default, it will make things a lot worse because it won't be the same unicode.
Re:Microsoft is not rebasing Windows to Linux
by AnIdiotOnTheNet on Monday October 12, 2020 @10:02AM
(#24754407)
Well, they say UTF-16, but they still allow unpaired surrogates in some places, so it is really more like UCS-2.
Re:Microsoft is not rebasing Windows to Linux
by chungy on Monday October 12, 2020 @12:06PM
(#24755875)
It is, Windows NT's adoption of Unicode predates UTF-16 and it's impossible to really make it UTF-16 compliant without breaking obscure corners of compatibility.
Most programs assume it's all UTF-16, and it happens to work most of the time... at least until you encounter those sequences that are invalid.
Re:Microsoft is not rebasing Windows to Linux
by ChrisSD on Monday October 12, 2020 @11:10AM
(#24755163)
Linux also allows invalid utf-8.
Re:Microsoft is not rebasing Windows to Linux
by JdeBP on Monday October 12, 2020 @11:31AM
(#24755433)
WTF-16 makes the distinction that invalid sequences are allowed.
Re:Microsoft is not rebasing Windows to Linux
by vbezhenar on Monday October 12, 2020 @10:21AM
(#24754580)
You can switch 1-byte encoding to UTF-8. But many applications probably won't work. And working applications probably are using unicode API anyway.
Re:Microsoft is not rebasing Windows to Linux
by GoblinSlayer on Monday October 12, 2020 @12:59PM
(#24756548)
It was explicit for a long time that ANSI is a legacy feature, and new API functions are often added without ANSI version, like CreateFile2.
Countless times that codepage thing has haunted me when moving data from Windows workstations into automatic processing systems hosted on Linux.