Re:Unitools – A suite of tools for working with Unicode in t...
by lmm on Wednesday March 02, 2016 @11:48AM
(#11211099)
If you're dealing with text from outside you have to handle encodings whether you like it or not. (I mean you can blindly copy bytes if you like, but you end up with corrupt data that you have no idea what it means, and may ultimately have to spend months or years recovering that information. I'm speaking from experience here).
Internally it very rarely matters what representation you use as long as you know what it is - pick one particular representation to use for all your internal functions and stick to it and you're fine. You shouldn't need to do any detection except at the boundaries, and efficiency concerns are usually overblown.
Internally it very rarely matters what representation you use as long as you know what it is - pick one particular representation to use for all your internal functions and stick to it and you're fine. You shouldn't need to do any detection except at the boundaries, and efficiency concerns are usually overblown.