News for hackers. Stuff that matters.
Re:Mercurial’s journey to and reflections on Python 3 by int_19h on Monday January 13, 2020 @02:44PM (#22037839)
> Python 2 had different types but you could be sloppy and it would kinda work out.

It would "kinda work out", if your Unicode strings were ASCII in practice, and only then. Because whenever a Unicode and a non-Unicode string had to be combined, it used ASCII as the default encoding to converge them.

Which is to say, it only worked out for English input, and even then only until the point where you hit a foreign name, or something like "naïve". Then you'd suddenly get an exception - and it happened not at the point where the offending input was generated, but at the point where two strings happened to be combined.

This was a horrible state of affairs for basically everybody except the English speakers, because there was a lot of Python code out there that was written against and tested solely on inputs that wouldn't break it like that.

Intermixing binary data with text can be represented just fine in a type system where the two are different. For your HTTP example, the obvious answer is that the values that are fundamentally binary, like the method name or the headers, should be bytes, while the parts that have a known encoding should be str - there's nothing there that requires actually mixing them in a single value. In those very rare cases where you genuinely do have something like Unicode followed by binary followed by Unicode in a single value, that is trivially represented by a (str, bytes, str) tuple.

The problem with the Python stdlib isn't that bytes and Unicode are distinct. It's that it's overly strict about only accepting Unicode in some places where bytes should be legal, too. This is orthogonal to them being separate types.

Re:Mercurial’s journey to and reflections on Python 3 by otabdeveloper4 on Tuesday January 14, 2020 @01:14AM (#22042302)
> Because whenever a Unicode and a non-Unicode string had to be combined, it used ASCII as the default encoding to converge them.

They could have just changed the default encoding to utf8. (For those too lazy to configure their Python properly.)

There, problem solved - and no need for a breaking Python 3.

Re:Mercurial’s journey to and reflections on Python 3 by int_19h on Tuesday January 14, 2020 @03:08AM (#22042800)
It would still be a mess any time you have to deal with byte strings that aren't UTF-8. The problem is with the implicit conversion itself - it shouldn't happen, because there's no way to properly guess the encoding. But there was no way to get rid of it without breaking things.
Re:Mercurial’s journey to and reflections on Python 3 by otabdeveloper4 on Tuesday January 14, 2020 @04:43AM (#22043272)
> But there was no way to get rid of it without breaking things.

Even such a breaking change would be a molehill compared to the mountain of breaking changes in Python 3.

Point is, they had one job, and they failed.

Re:Mercurial’s journey to and reflections on Python 3 by int_19h on Tuesday January 14, 2020 @12:26PM (#22046975)
That change was at the heart of the breaking changes around strings in Python 3. If the conversions remained implicit, most people would probably have never even noticed that string literals default to Unicode, or that some library functions now require Unicode strings.