UTF-8000: Unlimited UTF-8

Re:UTF-8000: Unlimited UTF-8 by flohofwoe on Monday September 21, 2026 @05:38AM (#49785091)

Yes, UTF-16 is the worst of all alternatives and should be abolished rather sooner than later.

UTF-32 is fixed-width for UNICODE code points, but a single visual character (e.g. a "grapheme cluster") can be built from multiple code points. This is separate from the encoding algorithm though, grapheme clusters are mostly a problem for the high level code working with already decoded text data (text rendering, comparison, sorting etc...).

[ Parent | on HN ]

Re:UTF-8000: Unlimited UTF-8 by nickyvdicarlo on Tuesday September 22, 2026 @02:07PM (#49805627)

I was working on optimizing a UUID scanner recently, and was baffled when I discovered that Java represents all char primitives in UTF-16, regardless of the original encoding.

So even given an ascii String, if you want to do something like

char c = input.charAt(index)

the JVM is going to jump to that index, check whether the character at that index isLatin(), then cast that single byte into to a 2 byte char... every single time.

in the naive solution, omething like 15% of the CPU cycles were spent checking isLatin() over and over again

[ Parent | on HN ]