News for hackers. Stuff that matters.
Re:Make JDK source code UTF-8 by josephg on Tuesday February 28, 2023 @11:13PM (#34978685)
I wonder if this can ever be fixed. If basically every Japanese organization disagrees with the Unicode consortium to the point that its undermining the use of unicode in entire countries, it might be worth unwinding CJK normalization.
Re:Make JDK source code UTF-8 by Freak_NL on Wednesday March 01, 2023 @04:41AM (#34980470)
This isn't the case obviously. Japanese organisations are involved in the Unicode process (something evidenced in the input for the several CJK Unified Ideograph Extensions). Thank to Unicode, many historical characters can be represented in digital text without resorting to placeholders or hacks.

Anecdotally, I've visited a group enthusiastically exploring the possibilities of the then-new CJK Unified Ideograph Extension B in Tokyo over a decade ago. The CJK issue mentioned here was a bigger problem then than now (most developers know how to handle it properly, e.g. with language declarations in HTML etc.).

People will use legacy encodings. That's just a fact of life. Sometimes out of inertia (i.e., it just works, so why bother changing it?), sometimes out of technical limitations of their pipeline, sometimes because of a lack of understanding, rarely out of principle. I'm stuck fixing an issue with some numb skull sending us SAML requests declared as UTF-8 containing some obsolete encoding instead, and this is for Dutch text! This just happens.

There is also nothing wrong with text in Shift-JIS, if that encoding encompasses all characters used.