News for hackers. Stuff that matters.
Re:The UTF-8-Everywhere Manifesto by pornel on Sunday April 29, 2012 @03:57PM (#3906878)
UTF-8 is easy to figure out. However, it's only a doorway to Unicode, and Unicode is not simple.

If you do anything more than chopping codepoints and passing them around — use a library. There's a lot of technical complexity due to quirks of Unicode and inherent complexity of world's diverse writing systems.

Avoid using concept of a "character" as much as you can, as it's fuzzy, e.g. there are combining characters and ligatures (codepoint != character).

Be aware that string comparison cannot be done just by comparing codepoints, and there are different levels of "sameness" of Unicode strings coming from different normalisations, e.g. NFC and NFKD.

Case-insensitive comparison cannot be done by lowercasing a string: http://www.moserware.com/2008/02/does-your-code-pass-turkey-...

Unfortunately I don't know much about RTL text, and there's lots of traps there too (e.g. there are control characters for controlling text direction).

There are no comments on this comment yet.