News for hackers. Stuff that matters.
Re:APFS is not safe to use with names which have Unicode nor... by pilif on Tuesday June 06, 2017 @07:58AM (#14496538)
HFS+ was the only file system I know of that was doing Unicode normalisation and is certainly was the only one choosing NFD which encodes characters in a way that's impossible for someone to easily type in the UI.

Short-term this will cause a lot of inconsistencies with applications using low-level APIs as the files currently existing will be NFD normalised, but any user-given path will very likely be more or less equivalent to what NFC would do.

So in the short term, this will be a mess (unless the APFS conversion on install-time also does a once-time conversion of the normal form), but in the long term, I believe this is the way to go.

And for that matter, the same should happen with regards to case-insensitivity which is even worse as ensuring case-insensitivity might actually be dependent on locale, which means that depending on the user's locale two file names might or might not be identical under case-insensitivity rules.

Unfortunately, it look like there's just too much legacy code around that plain doesn't work with a case-sensitive file-system on the mac (I'm looking at you, Adobe).

BTW: On my 10.13 Beta 1 setup with an APFS converted boot drive, unless I manually create a NFD encoded file name on the command line (which you can't do accidentally), everything is NFC both in the UI and on the command line. This also applies to files I haven't touched since the conversion to APFS.

Re:APFS is not safe to use with names which have Unicode nor... by xoa on Tuesday June 06, 2017 @11:29AM (#14497969)
>HFS+ was the only file system I know of that was doing Unicode normalisation

I've seen something in a couple comments now to this effect, does nobody here do anything with ZFS at all? It's a pretty great filesystem on not just illumos but FreeBSD, Linux, and macOS, and it has full native normalization support (and for all 4 forms too). Normalizing is optional, but it's definitely there and with Macs I have been using NFD for compatibility purposes for 5 years now with no discernible performance problems in day-to-day use. I know ZFS isn't remotely a majority but I don't think it's that obscure either.

Re:APFS is not safe to use with names which have Unicode nor... by cryptonector on Tuesday June 06, 2017 @12:34PM (#14498572)
Thank you for pointing this out.

In particular ZFS has normalization-preserving/normalization-insensitive behavior, which is far superior to HFS+'s opinionated normalization-on-create (to a form that is different from the common input modes' output!).

Re:APFS is not safe to use with names which have Unicode nor... by jorangreef on Wednesday June 07, 2017 @05:22AM (#14504771)
Yes, HFS+ implements normalization-insensitive behavior through not being normalization-preserving, in the same way that some FATs might implement case-insensitive behavior through not being case-preserving. They didn't realize that you could have both features: normalization-preserving and normalization-insensitive.
Re:APFS is not safe to use with names which have Unicode nor... by cryptonector on Wednesday June 07, 2017 @03:31PM (#14509221)
I guess it was a very forgivable lack of imagination. When we came up with form-preserving/insensitive we were motivated in great part by the interop mess caused by HFS+ -- we might not have arrived there without that mess, though I'd like to believe that someone would eventually have reached this conclusion regardless.
Re:APFS is not safe to use with names which have Unicode nor... by cryptonector on Tuesday June 06, 2017 @01:31PM (#14499122)
Not to repeat myself too much, but... ZFS also supports normalization. Only instead of normalizing on create, it has normalization-preserving/insensitive behavior.