News for hackers. Stuff that matters.
Re:Mercurial’s journey to and reflections on Python 3 by pdonis on Monday January 13, 2020 @05:54PM (#22039660)
> the teams insistence that file names were byte strings was the cause of lots of bugs when it came to Unicode support

File names are a different problem because Windows and Unix treat them differently: Unix treats them as bytes and Windows treats them as Unicode. So there is no single data model that will work for any language.

Re:Mercurial’s journey to and reflections on Python 3 by hsivonen on Tuesday January 14, 2020 @02:01AM (#22042509)
The Rust standard library has a solution for this that actually works: On Unix-like systems file paths are sequences of bytes and most of the time the bytes are UTF-8. On Windows, they are WTF-8, so the API users sees a sequence of bytes and most of the time they match UTF-8.

This means that there's more overhead on Windows, but it's much better to normalize what the application programmer sees across POSIX and NT while still roundtripping all paths for both than to make the code unit size difference the application programmer's problem like the C++ file system API does.

Re:Mercurial’s journey to and reflections on Python 3 by pdonis on Tuesday January 14, 2020 @05:25PM (#22049872)
> On Windows, they are WTF-8

Seems like an apt acronym for Windows... :-)

On a more serious note, Python seems to have done something fairly similar with the pathlib standard library module.

Re:Mercurial’s journey to and reflections on Python 3 by simias on Monday January 13, 2020 @06:06PM (#22039761)
Not to mention case-sensitivity issues. Can you have two files, one named "FILE.txt" and the other "file.txt" in the same directory for instance?
Re:Mercurial’s journey to and reflections on Python 3 by SSLy on Monday January 13, 2020 @06:40PM (#22040043)
On windows? Of course you can.
Re:Mercurial’s journey to and reflections on Python 3 by edgyquant on Monday January 13, 2020 @07:53PM (#22040639)
I'm certain you can on Linux as well. Only Macs old HFS would not allow it.
Re:Mercurial’s journey to and reflections on Python 3 by cataflam on Monday January 13, 2020 @08:10PM (#22040741)
Isn't this a fairly recent change?
Re:Mercurial’s journey to and reflections on Python 3 by amaranth on Tuesday January 14, 2020 @12:18AM (#22042070)
NTFS has always been case sensitive, Windows API just lets you treat it as case insensitive. If you pass `FILE_FLAG_POSIX_SEMANTICS` to `CreateFile` you can make files that differ only in case.
Re:Mercurial’s journey to and reflections on Python 3 by mathw on Tuesday January 14, 2020 @03:56AM (#22043009)
Good luck using those in some tools which use the API differently though. Windows filenames are endless fun. What's the maximum length of the absolute path of a file? Why, that depends on which API you're using to access it!
Re:Mercurial’s journey to and reflections on Python 3 by rurban on Wednesday January 15, 2020 @12:39AM (#22052001)
Even worse on Unix where it depends on the mount type. Haven't seen much proper long filename support in Unix apps or libs, it's much better in Windows land. Garbage in garbage out is also a security nightmare as names are not identifiable anymore. You can easily spoof such names.