News for hackers. Stuff that matters.
Re:Unicode is harder than you think by jmclnx on Tuesday July 25, 2023 @03:45PM (#36868274)
No kidding, you have not lived until you try and explain UTF-8 to people who only believes in what they called "doublebyte".

You think they get it, but surprise happens when a database load fails when loading Chinese Character "string" into a field sized calculated based upon 2 bytes per character.

Re:Unicode is harder than you think by theamk on Tuesday July 25, 2023 @04:13PM (#36868672)
Thank god for emojis! Those people would say, "No one in our org would use chinese" and refuse to fix things... but now I just point them to latest message from upper management which contain emoji or two.

(And emoji are such a fine example - once they ate on the table, you need support for combining characters, characters outside of BMP, ligatures.. a large part of Unicode spec)

Re:Unicode is harder than you think by throwawaymobule on Wednesday July 26, 2023 @10:48AM (#36878614)
It's a very strange series of events that led us here, with various stages of 'oh no' and 'oh yes' events happening.

Like Apple trying to region-lock emojis, or whoever decided merging CJK together was a good idea, or that plan9 dev who scribbled out UTF8 on a napkin.

Re:Unicode is harder than you think by qalmakka on Tuesday July 25, 2023 @04:14PM (#36868687)
It's terrible, and we IMHO owe that to some introductory university courses to Java (plus some Win32 veterans). I got very close to being rejected by a professor that was obstinately convinced that Unicode "characters" were 2 bytes because it drunk the Kool Aid in the '90s about Java's `char` type representing a Unicode character. Ugh. I still get angry by thinking back at that sometimes
Re:Unicode is harder than you think by jmclnx on Tuesday July 25, 2023 @06:44PM (#36870750)
I can relate, I remember a teacher stating "you never have to worry about the amount of memory". This was in the late 90s, I then asked "So I can load a 20 gig data file into memory", he said yes.
Re:Unicode is harder than you think by Dylan16807 on Tuesday July 25, 2023 @10:41PM (#36872512)
I'd say that particular answer is fine, because of mmap.