That was not what this article measured. UTF-8 and UTF-16 compressed to virtually the same size.
In the global Internet, UTF-16 and UTF-8 are the only games in town.
Then UTF (and EUC's) are the way to go.
It's not like you have to use the same encoding all the time.
Then you are going to feed someone garbage. Why feed people garbage?
Not if you know what you're doing. Not any more than using utf8 exclusively all the time and for all purposes.
This is nice in theory. In practice, people make mistakes. Make it easy on yourself.
> Not any more than using utf8 exclusively all the time and for all purposes.
Maybe I was unclear: Feeding me Chinese text in UTF-8 is not garbage. Feeding me anything in one of the GB encodings is garbage.
Garbage, to me, is text in an encoding I can't handle. If you only use UTF-8, that cannot possibly happen.
For instance: EUC-JP in Japan, BIG5 in Taiwan, GB in China, etc. Different EUC encodings are variable-length and are a lot more efficient since they put common subsets in lower parts of the table for each language, close to each other so they also compress better, while allowing tricks for text matching and searches (not really necessary for web sites, but it's nice to use the same encoding throughout applications sometimes). Russian and Greek are also basically multiplied by two in size.
There are a lot of other considerations.
If you think a 30%+ saving in size (and latency) is not a big deal, then you're a lot more likely to lose to local competitors. Note that gzipped or otherwise compressed text makes differences even worse at least in the case of Japanese - where UTF-8 text gets de-aligned all over the place to odd byte sizes and compresses worse. Add to that the fact that Asians browse the net A LOT from the phone and have done so for much longer than westerners and in a bigger percentage, and you have your problem exacerbated even further.
There is a lot more to consider and like it or not it's not as simple as "UTF-8 for everything and everybody, ever!"