Re:UTF-8 is a brilliant design
by adgjlsfhk1 on Friday September 12, 2025 @04:57PM
(#45226684)
python does (although it will use 8 or 16 bits per character if all characters in the string fit)
Re:UTF-8 is a brilliant design
by Mikhail_Edoshin on Friday September 12, 2025 @11:45PM
(#45229131)
No, UTF-16 is much simpler in that aspect. And its design is no less brilliant. (I've written an state machine encoder and decoder for both these encodings.) If an application works a lot with text I'd say UTF-16 looks more attractive for the main internal representation.
UTF-16 is simpler most of the time, and that's precisely the problem. Anyone working with UTF-8 knows they will have to deal with multibyte codepoints. People working with UTF-16 often forget about surrogate characters, because they're a lot rarer in most major languages, and then end up with bugs when their users put emoji into a text field.
UTF-32 would be a fair comparison, but it is 4 bytes per character and I don't know what, if anything, uses it.