Re:Plain Text Doesn’t Exist: Unicode and encodings demystified
by stormbrew on Monday August 26, 2013 @01:01PM
(#6278257)
> This is exactly my point: lots of code also treats utf-8 as if it were ASCII.
Two things to this:
- In many cases this is an entirely non-destructive mistake to make. This is an advantage of utf-8, that intermediaries that only care about ascii characters (even in the presence of extended characters) can work with it.
- It will become readily apparent that you've got a problem on the first attempt to port your app to any language with a non-ascii script. With UTF-16 it only becomes apparent when you go beyond the BMP.
This is exactly my point: lots of code also treats utf-8 as if it were ASCII.
> how often do you see bulk pure text, without any markup, and that can't be compressed?
That's a good argument. Thanks :)