Re:APFS does not normalize Unicode filenames
by askvictor on Saturday March 25, 2017 @05:29AM
(#13954756)
Google drive allows you to have identically named files in the same directory. Doesn't quite map onto a desktop filesystem, but the filename is not a unique identifier, but just a convenience for the user or application.
Re:APFS does not normalize Unicode filenames
by jameshart on Saturday March 25, 2017 @08:22AM
(#13955271)
Even with normalization, you can just sprinkle an arbitrary selection of non breaking zero width space characters into a string and create an infinite variety of identical-looking but distinct codepoint sequences. Treating filenames as bundles of bytes is fine, but at that point it seems dangerous to assume that interpreting them as encoded strings is a valid way of distinguishing them.
Re:APFS does not normalize Unicode filenames
by rurban on Saturday March 25, 2017 @04:33AM
(#13954628)
You are talking about confusables.
Unfortunately normalization cannot handle this. Only certain apps treat confusables as confusables: email, web browsers, dns services, usually via libidn.
Re:APFS does not normalize Unicode filenames
by kbenson on Saturday March 25, 2017 @04:34PM
(#13957442)
> For that matter, even if you were limited to plain keyboard ASCII, you could have two really long filenames that only differ in one character, which are effectively indistinguishable without massive hair-pulling.
My initial response was to not present confusing data to the user, but then I thought of exactly this. In many fonts, lowercase "L" (l) and uppercase "i" (I) look the same. I'm sure in some fonts they actually are identical, rather than just being nearly indistinguishable to the human eye.
To the discussion is not about whether we should present confusing identifiers to users, but at what point if it becomes more common does it cause enough of a problem to be a bad choice, and what techniques are there to mitigate it?
Show two files with apparently identical names. Is that so surprising? There are many many ways for two different Unicode strings to look visually identical (or near-identical) even if they aren't equivalent under normalization. For that matter, even if you were limited to plain keyboard ASCII, you could have two really long filenames that only differ in one character, which are effectively indistinguishable without massive hair-pulling. If you want to be robust, there's no way around having means to identify files other than names.
(I agree with your sentiment about tagging.)