News for hackers. Stuff that matters.
Re:Unitools – A suite of tools for working with Unicode in t... by TazeTSchnitzel on Wednesday March 02, 2016 @10:56AM (#11210695)
While you can't represent that as plain ASCII, I think Windows-1252, an 8-bit single-byte encoding, can handle that quite well.
Re:Unitools – A suite of tools for working with Unicode in t... by kstrauser on Wednesday March 02, 2016 @01:01PM (#11211674)
That's an excellent point. Let's generalize that Windows-centric idea of a code page to something that can handle most/all known languages: a Universal Coded character set, if you will.
Re:Unitools – A suite of tools for working with Unicode in t... by pekk on Wednesday March 02, 2016 @11:00AM (#11210725)
I guess we can get some arbitrary text, then check whether all its characters can be encoded as Windows-1252, store it with a note that it is in Windows-1252, and then check that note before decoding it. Clearly so much better than UTF-8.
Re:Unitools – A suite of tools for working with Unicode in t... by TazeTSchnitzel on Wednesday March 02, 2016 @11:05AM (#11210756)
They tried: https://en.wikipedia.org/wiki/ISO/IEC_2022

It was... horrible.

Re:Unitools – A suite of tools for working with Unicode in t... by jquast on Wednesday March 02, 2016 @12:25PM (#11211382)
There is an art in discovering which encoding the text vs. represented by application, by making sense of nonsense.

https://jeffquast.com/soviet-encodings.jpg

Re:Unitools – A suite of tools for working with Unicode in t... by masklinn on Wednesday March 02, 2016 @11:48AM (#11211097)
Sure I mean who wants to visit Госуда́рственный Эрмита́ж or eat 삼겹살? Moonspeak's not really text right?