News for hackers. Stuff that matters.
Re:NumPy 1.16 is the last release to support Python 2.7 by coldtea on Monday January 21, 2019 @03:07AM (#18958610)
>Just rip the band-aid off. Some of the async stuff in Python 3 is awesome.

Will that awesome async stuff help us migrate several 100s of thousands of lines of 2.7 code?

Re:NumPy 1.16 is the last release to support Python 2.7 by pjmlp on Monday January 21, 2019 @03:15AM (#18958656)
I guess 10 years was more than enough time, maybe 2.x series will be the Cobol of Python world.
Re:NumPy 1.16 is the last release to support Python 2.7 by coldtea on Monday January 21, 2019 @06:08AM (#18959509)
10 years where the first P3K versions were unusable and the rest still didn't have tons of dependencies?

Yeah, right. There's a reason even companies with tons of resources like Google still use 2.7.

Re:NumPy 1.16 is the last release to support Python 2.7 by sametmax on Monday January 21, 2019 @06:29AM (#18959606)
12 years. It's all been very usable for 6 years. So you had 6 years to use the six or python-future lib, and 6 years to remove it. 100k lines, it's a 2 weeks job if your code is properly tested. It's quite ok given python is 25 years old and asked for it once.

But good news, you don't have to do it. Python 2.7 will still work after 2020. We, the community, will just stop working on it for free. You can start paying us, or do the job for free like we did since 1991.

You can also migrate to another tech, but they will break things as well, and they won't give you 12 years to move. Ruby and node gave 2 years. Node forked 2 times. And it's a google based tech.

I'm tired of hearing complains, given how good you have it.

Re:NumPy 1.16 is the last release to support Python 2.7 by coldtea on Monday January 21, 2019 @10:05AM (#18960801)
>12 years. It's all been very usable for 6 years. So you had 6 years to use the six or python-future lib, and 6 years to remove it. 100k lines, it's a 2 weeks job if your code is properly tested

It took Flask several years, complaints, and a custom change for their use case, to port over.

Some people have actual production code which they don't rewrite for the fun of it (and even less with comically underestimated "2 weeks per 100K" runs), because the core decided a non-backwards compatible version is the future.

>But good news, you don't have to do it. Python 2.7 will still work after 2020. We, the community, will just stop working on it for free. You can start paying us, or do the job for free like we did since 1991.

Well, you're not the "community", at best you're one committer. The community (not necessarily the core devs) will fork and maintain way beyond 2020, and it will be for free too.

>I'm tired of hearing complains, given how good you have it.

Sorry, didn't know people must walk on tiptoes lest they tire you with their complains...

Re:NumPy 1.16 is the last release to support Python 2.7 by LeanderK on Monday January 21, 2019 @10:42AM (#18961050)
Well, programming languages evolve and to maintain a clean language (which python aspires to be), it has to cut some ties.

I think the overly backward-compatible way in which the change was managed really prolonged the adtoption and getting python 3.x to mature. The Python-Community was hanging between those versions far too long and it really hurt the ecosystem and the whole python-experience.

> Some people have actual production code which they don't rewrite for the fun of it (and even less with comically underestimated "2 weeks per 100K" runs), because the core decided a non-backwards compatible version is the future.

Of course, but you (probably) still have to maintain the code. And it's not a full rewrite, more an adaption. I find most changes to be quite mechanical. Also, it's not the python-comunities fault, you chose python and you're getting probably using it for some business purpose. So i think the python community can expect you to maintain it, including language adaptions. You can still ignore this and don't update, but libraries will probably start getting incompatible and there will come a time when things break. Wheter that's important enought to migrat is up to you.

I don't think it's fair to just complain loudly, after all, the python community is not expection a full rewrite. They have given enough time to migrate, i think took much, but it's still a lot. Being completely backward compatible forever was never promised, and i think is an complete anti-feature. Write once and never think about it is not the python-way, but having a clean, smart, elegant language. It also really hurts java, which just walks with all the backage of the past and glacial speed for language-updates, which probably is required when everything you do is going to be kept forever in the language.

Re:NumPy 1.16 is the last release to support Python 2.7 by sametmax on Monday January 21, 2019 @10:09AM (#18960838)
Says the person who has benefited from it for free for 25 years and doesn't have to pay the price for maintening it even further.
Re:NumPy 1.16 is the last release to support Python 2.7 by sweettea on Monday January 21, 2019 @07:11AM (#18959812)
I'm a big fan of Tauthon, a backwards-compatible Python 2.7 fork with some features backported from Python 3. Personally I've yet to see a Python 3 feature that would be useful in my code.
Re:NumPy 1.16 is the last release to support Python 2.7 by sametmax on Monday January 21, 2019 @08:01AM (#18960074)
After 15 years of python, going from client to client, my experience is the opposite: i've yet to see a code base that would not benefit from python 3. Usually people saying that just don't know much about python 3.

However, I understand that those benefits might not be motivation enough to migrate in some contexts.

In those cases, I advise to freeze the project in time, isolating it by vendoring dependancies and either compile it with nuitka or distribute it in docker.

Re:NumPy 1.16 is the last release to support Python 2.7 by mrweasel on Monday January 21, 2019 @06:56AM (#18959731)
The issue was that some people didn't even start writing Python 3 compatible code.

I would go so far as to say that Python 3.4 was the first one you could really use in production, that leaves you with 5 years to handle the upgrade. Still before that you should at least have started to add basic stuff like importing print_function and at testing all new stuff against Python 3. Many just kept ignoring Python 3 and wrote line after line of new incompatible Python 2 code.

I no longer feel sorry of those who are stuck on Python 2.7 and now have less than a year to upgrade. At this point it's your own fault.

Re:NumPy 1.16 is the last release to support Python 2.7 by brylie on Monday January 21, 2019 @03:16AM (#18958659)
What percentage of those lines do you estimate will need changes?
Re:NumPy 1.16 is the last release to support Python 2.7 by walrus1066 on Monday January 21, 2019 @03:29AM (#18958719)
Imagine that's part of the challenge. Since python is dynamically typed/duck typed, it's hard to audit all usages of a variable. I.e. If I see a py2 string declared, how can I be 100% sure I don't break downstream components by making it a bytes or py3 unicode string? Some might be using it as the former, others the latter.

You'd need insane test coverage, of all possible code paths, to be 100% confident.

Re:NumPy 1.16 is the last release to support Python 2.7 by wenc on Monday January 21, 2019 @09:48AM (#18960684)
I was responsible for the Python 2 to 3 migration for various code bases at work. I used 2to3 to check for issues, but did most of the code rewrites by hand.

Most of the conversion difficulties were related to Unicode/text/binary string handling in Python 3, and 2to3 didn't catch all of them. This was the biggest challenge of the entire process. Stuff would fail in production because due to improper string handling. Python 2 was remarkably permissive (i.e. loosy goosey), whereas Python 3 is stricter and arguably more correct, but this strictness has a cost.

The other class of problems is the restructure of certain std libs, like urllib. We took the opportunity to move away from "urllib" to "requests".

This site [1] was invaluable in understanding the migration issues.

All in all, apart from breaking Unicode issues, the migration process was fairly easy.

[1] http://python3porting.com/problems.html

Re:NumPy 1.16 is the last release to support Python 2.7 by CJefferson on Monday January 21, 2019 @05:54AM (#18959430)
The problem is which lines. I found code all over the place which broke for unicode reasons (we were reading in the output of GCC, and printing it back out, and it didn't work any more as GCC doesn't output valid UTF8 in some cases).

In the end I decided moving to Rust was less work than Python 3 (in terms of being sure my program was reliable and wouldn't error out with unicode errors).

Re:NumPy 1.16 is the last release to support Python 2.7 by ric2b on Monday January 21, 2019 @06:28AM (#18959598)
> I found code all over the place which broke for unicode reasons (we were reading in the output of GCC, and printing it back out, and it didn't work any more as GCC doesn't output valid UTF8 in some cases).

That means the code was working with raw bytes, not utf-8 strings, so that's what you should convert to on Python3.

That means using `.encode()` and `.decode()` or using bytestrings.

Python3 doesn't break string usage, it makes you do it correctly. Expect a lot more of that kind of pain by moving to Rust (not saying it's not worth, you'll indeed get more correct programs done).

Re:NumPy 1.16 is the last release to support Python 2.7 by CJefferson on Monday January 21, 2019 @06:48AM (#18959690)
The problem I had with python is that (because it is dynamically typed), and quite a few methods take both bytes and strings, it can be tricky to track what you have and often things don't break until you feed in a non-UTF8 string.

I found it much easier in Rust, as the two types are just distinct, and an incorrect program just fails to compile.

Re:NumPy 1.16 is the last release to support Python 2.7 by orf on Monday January 21, 2019 @09:44AM (#18960659)
In Python 3 they are two distinct types as well.
Re:NumPy 1.16 is the last release to support Python 2.7 by lmm on Monday January 21, 2019 @11:04AM (#18961212)
They're not really "types" when you only find out at runtime.
Re:NumPy 1.16 is the last release to support Python 2.7 by joshuamorton on Monday January 21, 2019 @03:11PM (#18963204)
When handling tricky string issues, my first step is usually to explicitly type annotate the code (using mypy annotations). This makes string handling and tracking much simpler.
Re:NumPy 1.16 is the last release to support Python 2.7 by orf on Monday January 21, 2019 @11:10AM (#18961263)
Sure they are.
Re:NumPy 1.16 is the last release to support Python 2.7 by speedplane on Monday January 21, 2019 @03:22AM (#18958688)
100% will need to be tested.
Re:NumPy 1.16 is the last release to support Python 2.7 by coldtea on Monday January 21, 2019 @06:09AM (#18959512)
Anything dealing with text for starters.