Re:The UTF-8-Everywhere Manifesto
by daliusd on Sunday April 29, 2012 @01:33PM
(#3906452)
"One of the advantages of UTF-16 is knowing right away it's UTF-16 as opposed to deciding if it's UTF-8/ASCII/other encoding."
It is actually not that simple. By using UTF-16 you already have at least two problems:
1. You should know if byte order is big endian or little endian.
2. You should know if your API supports whole unicode set or only 65536 symbols. E.g. Windows API. Do you know answer? What will happen if your user wants to abuse your system by using symbols outside those 65536.
It is actually not that simple. By using UTF-16 you already have at least two problems:
1. You should know if byte order is big endian or little endian.
2. You should know if your API supports whole unicode set or only 65536 symbols. E.g. Windows API. Do you know answer? What will happen if your user wants to abuse your system by using symbols outside those 65536.