UTF-8 Everywhere (2012)

Re:UTF-8 Everywhere (2012) by asveikau on Sunday June 19, 2016 @10:48PM (#11935806)

1 grapheme = 1 unit of datatype is a good start. There is a lot of complexity in Unicode, but the mental model of a "char" is what most developers are thinking, largely blind to that complexity. Kudos to anyone trying to make that more "common sense" model closer to reality.

However... How does a language or library attempting to abstract this part (like swift might) deal with the other, unrelated annoying aspects of Unicode? Even if 1 glyph is 1 "char", and we normalize all the inputs, there is still, say ... Bi-di text.

[ Parent | on HN ]

Re:UTF-8 Everywhere (2012) by johncolanduoni on Monday June 20, 2016 @12:49AM (#11936106)

I don't think there is an easy answer to this. Even just pure RTL text is hard to support: IIRC Android and iOS only recently made all the built-in views support it. This is complicated by the fact that most developer teams don't have nor can afford to hire someone familiar with RTL to implement this kind of stuff.

[ Parent | on HN ]

Re:UTF-8 Everywhere (2012) by asveikau on Monday June 20, 2016 @01:10AM (#11936171)

I don't think the rules in Unicode are simple for anyone. I have seen professional developers who are native speakers of RTL languages screw it up. All this stuff about the directionality of a paragraph and the embedded markers... It boggles the mind. But a truly international product should be getting it right. Sad mismatch there.

PS: bit of trivia that people forget these days, Win32 has been supporting it for longer than android/iOS.

[ Parent | on HN ]