News for hackers. Stuff that matters.
Re:Mojibake by zacharynewton on Tuesday October 04, 2022 @09:05PM (#33089644)
One of the great parts of the Python ecosystem for data processing is https://ftfy.readthedocs.io/en/latest/ which can handle mojibake and many other unicode-related translation problems.

But seriously, I'm always a little upset when data vendors/customers/etc don't specify the encoding they are using. You'd be surprised how many official or unique sources still use weird encodings in the name of compatibility.

Re:Mojibake by wiredfool on Wednesday October 05, 2022 @05:25AM (#33093134)
FTFY is amazing. Really useful for processing excel generated csvs.
Re:Mojibake by nness on Wednesday October 05, 2022 @10:09AM (#33095784)
Funny, I have used it for the same use-case (and a sad reminder how horrific Excel's handling of UTF-8 in CSV files can be...)
Re:Mojibake by xk3 on Friday October 14, 2022 @02:39PM (#33207261)
If you want to test ftfy online it's available here:

https://ftfy.vercel.app/