News for hackers. Stuff that matters.
Re:APFS does not normalize Unicode filenames by TazeTSchnitzel on Saturday March 25, 2017 @10:26AM (#13955794)
Confusable characters look similar or the same to humans.

Canonically equivalent Unicode sequences look the same to machines.

The latter is a much more significant problem, because it can wreak havoc with interoperability.

Re:APFS does not normalize Unicode filenames by acdha on Saturday March 25, 2017 @10:46AM (#13955901)
Very true - and it's amplified by inconsistency allowing problems to spread further before being noticed. At work I deal with a lot of Bag-It archives where we have a text manifest of checksums accompanying files on disk and this reliably bites users of tools when something (archive or network transfer tool, Git or SVN, transition to/from a Mac with HFS+, etc.) causes the encoding in the manifest not to match the local filesystem, and the confusion is amplified because some tools will handle normalization differences so the bug report is “why does tool A say this file is missing when Explorer/Finder and tool B say it's fine?”
Re:APFS does not normalize Unicode filenames by slrz on Saturday March 25, 2017 @09:54PM (#13958591)
Canonically equivalent Unicode sequences look the same to machines.

Memcmp disagrees, as do the default equality operators of most programming languages in existence.

Re:APFS does not normalize Unicode filenames by TazeTSchnitzel on Saturday March 25, 2017 @10:04PM (#13958641)
Sure, but normalisation can nonetheless happen automatically and implicitly in many places.
Re:APFS does not normalize Unicode filenames by vbezhenar on Sunday March 26, 2017 @03:59AM (#13959711)
Rust uses separate string type for file names. I think, that's a good approach. If language normalizes strings behind your back, that's not very good.