News for hackers. Stuff that matters.
Re:Code Page 437 Refuses to Die by tankenmate on Sunday May 08, 2016 @11:37AM (#11654439)
Indeed, this is exactly what the Linux kernel does; it treats filenames like a binary blob, with the exception that the bytes 0x2f (ASCII '/') and 0x00 (ASCII NUL) are not accepted (regardless of where they are in the byte string).
Re:Code Page 437 Refuses to Die by Kristine1975 on Sunday May 08, 2016 @12:20PM (#11654576)
> it treats filenames like a binary blob

Which can result in files with seemingly identical names when one UTF-8-encoded file name uses combining characters and the other one uses precomposed ones. Not to mention the fact that e.g. Å is contained in Unicode as both "latin capital letter A with ring above" and "Angstrom sign".

After you've solved encoding, next comes Unicode normalization ;-)

Re:Code Page 437 Refuses to Die by aleh on Sunday May 08, 2016 @01:15PM (#11654759)
It is not a big deal as long as seemingly identical file names are treated by OS as different.

Unicode normalization is not the only problem here, e.g. Latin 'a', 'e', 'T' are exactly the same as Cyrillic 'а', 'е', 'Т' in most fonts which makes it possible for two files to have seemingly same names even in some 8-bit encodings.

Re:Code Page 437 Refuses to Die by Hondor on Sunday May 08, 2016 @03:20PM (#11655339)
Even Latin 'I' and 'l' and the digit '1' are visually indistinguishable in some fonts! So are trailing spaces and different numbers of spaces. This is such a pervasive problem that maybe we can just give up and expect users to get used to it.
Re:Code Page 437 Refuses to Die by Hondor on Sunday May 08, 2016 @03:16PM (#11655322)
Wow, so characters like U+022F (ȯ) and U+042F (Cyrillic letter Я) U+062F (Arabic letter د) are not allowed but nearly everything else is? Some of those are letters used in actual languages. That's sure to make people scratch their heads.
Re:Code Page 437 Refuses to Die by rdancer on Sunday May 08, 2016 @03:55PM (#11655457)

    codepoint -> encoding in UTF-8

    U+022F    -> C8 AF  
    U+042F    -> D0 AF  
    U+062F    -> D8 AF
Remember, UTF-8 is self-synchronising: when you pick up at a random point within a stream, there is no ambiguity as to whether you are in the middle of a sequence or not. Valid lower codepoints appearing in the encoding of higher codepoints would violate this property.
Re:Code Page 437 Refuses to Die by pzb on Sunday May 08, 2016 @03:48PM (#11655421)
When encoding those characters in UTF-8, you will never end up with 0x2F as a byte. On of the properties of UTF-8 is that bytes with the high bit not set (e.g. 0x0 to 0x7e) never appear unless the are representing the 0 - 127 codepoints.
Re:Code Page 437 Refuses to Die by cataphract on Sunday May 08, 2016 @03:58PM (#11655468)
If you encode them in UTF-8, they are allowed. In UTF-8, to represent a given code point, only the first code unit can be in the range 0x00 to 0x7F. So if you're reading UTF-8 content and find a byte with value 0x00-0x7F, you can be sure it is a 1-byte long code unit sequence representing the code points U+0000 to U+007F.