News for hackers. Stuff that matters.
Re:APFS is not safe to use with names which have Unicode nor... by Piskvorrr on Tuesday June 06, 2017 @09:02AM (#14496863)
Indeed not: there is an almost infinite number of things that Latin-1 did not have; e.g. any characters beyond Western Europe. While keeping the 128-256 block compatible was a part of early Unicode (hence the "one-glyph" é), having composed characters was a Unicode primary design goal (hence e and the composing accent). A pure Unicode implementation would have been better, maybe; what we have instead is one that has gained mass adoption.

https://en.wikipedia.org/wiki/Latin-1_Supplement_(Unicode_bl...

Re:APFS is not safe to use with names which have Unicode nor... by rurban on Tuesday June 06, 2017 @11:57AM (#14498190)
The biggest actual problem is Greek vs Cyrillic. They appear the same, but use different encodings. Any filename (directory entry as identifier) must forbid mixed scripts. See TR31 http://www.unicode.org/reports/tr31/ Combining marks or RTL switching tricks are the other popular spoofs.
Re:APFS is not safe to use with names which have Unicode nor... by cryptonector on Tuesday June 06, 2017 @05:21PM (#14501457)
Confusables are a real problem.

Some Cyrillic and Greek letters even look like Latin letters. And in some fonts '1' (one) and 'l' (ell) look the same even in Latin.

Dealing with these is much harder than with normalization forms. And confusables are a source of serious security headaches.

Re:APFS is not safe to use with names which have Unicode nor... by Piskvorrr on Wednesday June 07, 2017 @04:56AM (#14504663)
1Il|!
Re:APFS is not safe to use with names which have Unicode nor... by kps on Tuesday June 06, 2017 @11:54PM (#14503626)

  > Any filename (directory entry as identifier) must forbid mixed scripts.
So I'm not allowed to name my files “A=πr²” or “10kΩ Резисторы”?
Re:APFS is not safe to use with names which have Unicode nor... by rurban on Wednesday June 07, 2017 @08:35AM (#14505776)
According to me not if you mixed Greek with Cyrillic, and not Math with Cyrillic. Unfortunately nobody cares about Unicode identifier security models. Garbage out is the most popular ideology.
Re:APFS is not safe to use with names which have Unicode nor... by kps on Wednesday June 07, 2017 @11:34AM (#14507255)
I take the view that modern file names are human readable labels, not access keys.

“10kΩ Резисторы” is Cyrillic, Latin, and Greek (U+2126 canonically maps to U+03A9). A proposal that disallows that but allows “10kΩ Resistors” is a political non-starter.

Re:APFS is not safe to use with names which have Unicode nor... by rurban on Wednesday June 07, 2017 @05:45PM (#14510107)
The Unicode consortium has published recommendations with various security levels everybody enjoys to ignore. Following even the weakest level would be a starter, but not in current filesystems community. This needs to start in a new OS or with high profile spoofing attacks, such as on GitHub.
Re:APFS is not safe to use with names which have Unicode nor... by nottorp on Tuesday June 06, 2017 @09:22AM (#14496983)
Well, composed characters may or may not have been a good idea, although they're a pain for text search if you ask me.

But why oh why did they have to specify FOUR normalization forms?

Ref: http://www.unicode.org/reports/tr15/#Norm_Forms

Re:APFS is not safe to use with names which have Unicode nor... by cryptonector on Tuesday June 06, 2017 @04:43PM (#14501146)
Pre-composed characters were needed to make transcoding to ISO-8859-* fast.

Also, for Hangul, though decomposition is the better form to use normally (since it's phonetic, not syllabic), conversion to pre-composed syllabic form is very useful sometimes.

Even without pre-composition there would have been multiple equivalent forms of writing any character that requires more than one combining codepoint.

The only way to have avoided normalization altogether would have been to have no combining codepoints, and only a complete set of pre-compositions. This would have been less flexible, and very obnoxious for Hangul and possibly others.

I believe the need for normalization was simply unavoidable. It's not the fault of Unicode but the fault of humans' script designs. And it's OK.

Re:APFS is not safe to use with names which have Unicode nor... by the_mitsuhiko on Tuesday June 06, 2017 @10:48AM (#14497617)
> But why oh why did they have to specify FOUR normalization forms?

Ignoring compatibility normalizations there are really only two and neither of those two you can remove. So that's a bit of an odd question. If you want to remove the compatibility forms that's fair but nobody is force to use them. You can consider them "partially normalized" or "not normalized at all" for all intents and purposes

Re:APFS is not safe to use with names which have Unicode nor... by cryptonector on Tuesday June 06, 2017 @04:44PM (#14501153)
You can ignore the K forms. So that's two forms, really, NFC and NFD.
Re:APFS is not safe to use with names which have Unicode nor... by TazeTSchnitzel on Tuesday June 06, 2017 @10:23AM (#14497401)
Because you need a normalisation form fit for your particular purpose.
Re:APFS is not safe to use with names which have Unicode nor... by Piskvorrr on Tuesday June 06, 2017 @09:24AM (#14496992)
¯\_(ツ)_/¯