News for hackers. Stuff that matters.
Re:The Python Unicode Mess by ddevault on Saturday October 06, 2018 @10:58AM (#18155442)
Filenames aren't text, they're bytes. You shouldn't decode them except if you need to for display (and in that case, you should be prepared to handle filenames which cannot be decoded as text). You should decode data into it's canonical representation as early as possible and encode it as late as possible, but the canonical representation of a filename is not text.
Re:The Python Unicode Mess by bartwe on Saturday October 06, 2018 @12:16PM (#18155838)
Filenames are text if you ever want to let a user interact with them by name.
Re:The Python Unicode Mess by ProblemFactory on Sunday October 07, 2018 @02:48AM (#18159184)
The two conditions

* OS treats filenames as bytes and allows arbitrary byte strings,

* Users want to edit filenames as text

are in conflict and you can't solve it without forcing some encoding or giving them a byte editor. The solution "filenames are bytes but we'll treat it as ASCII because 0x32-0x7E should be enough for anybody" is not acceptable for möst of thé wõrld.

Re:The Python Unicode Mess by Sir_Cmpwn on Saturday October 06, 2018 @12:20PM (#18155850)
I addressed this in my original comment.