News for hackers. Stuff that matters.
Re:Node Modules at War: Why CommonJS and ES Modules Can’t Ge... by masklinn on Thursday August 06, 2020 @06:10AM (#24069601)
> Porting Python 2 code to Python 3 was never the problem.

You're, frankly, high as a kite. As the author of one such port on a large codebase, porting Python 2 code to Python 3 was absolutely a problem.

The issue with 2to3 is that it assumed you'd do the conversion as a one-shot and straight flip over with entirely separate codebases (or dropping Python 2 entirely) from there on, which is completely insane.

What the vast majority of transitioning projects needed was a way to migrate progressively through cross-version codebases:

* libraries weren't going to drop older Python versions, the very few which attempted that (I'm aware of dateutil at least) were extremely painful to users and IIRC ended up rolling back to a more progressive codebase

* programs being distributed were not going to drop Python 2 until EOF at least

* large codebases simply could not afford to perform a completely untested and uncheckable one-shot transition

That's why the community created cross-version support tools like six and friends, and lobbied hard for the reintroduction of cross-version features into Python 3 e.g. reintroduction of `callable` (3.2), reintroduction of the "u" prefix (3.3), reintroduction of % on bytes (3.5) and I'm sure others.

> Again, there are exceptions, but the vast majority of websites or data processing scripts or whatnot are significantly re-written or decommissioned in – pulling a number out of my arse – five years or so. […] Python 3 because it would be incompatible with the Python 2 ecosystem

What do you think "the python 2 ecosystem is" except for a ton of projects which needed to be ported from Python 2 to Python 3 exactly? "the vast majority of websites or data processing scripts" is not an ecosystem, tooling and libraries are.

Do you somehow think numpy and lxml and django just scrapped their entire Python 2 codebase and rewrote the entire thing from scratch? Of course not.

> That problem could have been prevented with a 3to2 tool, which would compile the Python 3 code to Python 2, allowing it to be used with the existing ecosystem.

3to2 would not have worked any better better than 2to3. The syntactic incompatibilities between the version were not the hard part, the semantic ones were, and 3to2 would not have succeeded any better than 2to3 there because it simply couldn't.

All 3to2 would have given you is a broken Python 2 codebase from a Python 3 codebase you could not even run.

Re:Node Modules at War: Why CommonJS and ES Modules Can’t Ge... by nailer on Thursday August 06, 2020 @07:57AM (#24070140)
> You're, frankly, high as a kite.

https://news.ycombinator.com/newsguidelines.html

"Be kind. Don't be snarky. "

Re:Node Modules at War: Why CommonJS and ES Modules Can’t Ge... by kqr on Thursday August 06, 2020 @08:17AM (#24070245)
Isn't "migrating progressively through cross-version codebases" exactly what a 3to2 compiler would allow you to do? You'd write new code (and incrementally port old code) to Python 3, and then when you build the project you'd compile the Python 3 code down to Python 2 before building the entire project as Python 2, including all dependencies.

So for over a decade, you'd be writing Python 3 code but building a Python 2 project. After a decade, one would hopefully have managed to port everything over to Python 3, and can then drop the "compile to Python 2" step.

A 3to2 tool would solve exactly the problems you're bringing up, with no intoxication needed!

----

I don't see why 3to2 would not have worked. It's a compiler. We have written compilers before. We can write compilers. Compilers work. Both languages are Turing complete and about the same in expressiveness, so clearly compiling one to the other is something we can do.

What we cannot (reasonably) do is produce nice, human-looking target code from a compiler. This is what 2to3 attempted to do, and what a 3to2 would never need to do.

Who cares if the Python 2 code it produced looked a little wonky, as long as it was correct and roughly recogniseable as being built from the original Python 3 code? The Python 3 code is the one humans write and edit. The Python 2 code is just compiler output to make it compatible with all of the Python 2 ecosystem until one is ready to go fully Python 3.

----

This is also not something completely foreign and unheard of. This is exactly how I've been porting legacy JavaScript applications to a more modern ECMAScript approach: write new code in the modern way, compile it down to legacy code compatible with the existing legacy code, and then run that.

Slowly but surely, more and more things are being written in the modern fashion, and eventually, when everything has caught up, the compatibility conversion to the legacy code can be dropped.

Re:Node Modules at War: Why CommonJS and ES Modules Can’t Ge... by masklinn on Thursday August 06, 2020 @11:40AM (#24071924)
> Isn't "migrating progressively through cross-version codebases" exactly what a 3to2 compiler would allow you to do?

No. You'd have to migrate everything to Python 3 at once, and then "maintain" two different and incompatible codebases, one generated from the other.

> A 3to2 tool would solve exactly the problems you're bringing up, with no intoxication needed!

It would half solve (at best) one of them.

> I don't see why 3to2 would not have worked. It's a compiler.

Unless your 3to2 would literally reimplement Python 3 in Python 2, it would be a translator / bridge. Which can't work due to the semantics difference between Python 2 and Python 3: not every change translates mechanically and reliably. If they did, 2to3 would have worked.

> Who cares if the Python 2 code it produced looked a little wonky,

The maintainer who has to debug, probably.

> as long as it was correct

Which it wouldn't be.

> This is also not something completely foreign and unheard of. This is exactly how I've been porting legacy JavaScript applications to a more modern ECMAScript approach: write new code in the modern way, compile it down to legacy code compatible with the existing legacy code, and then run that.

The "modern way" of javascript is additive, it's about adding new features to the language, some of which can be implemented in terms of the old one (and which you can thus "backport" through a compiler or extending the existing APIs).

You don't run into pieces of code which are syntactically identical but semantically divergent. Which you absolutely do in P2 v P3.

Re:Node Modules at War: Why CommonJS and ES Modules Can’t Ge... by kqr on Thursday August 06, 2020 @12:40PM (#24072649)
> You'd have to migrate everything to Python 3 at once, and then "maintain" two different and incompatible codebases, one generated from the other.

Absolutely not. Well, you could, but the better approach would be to write individual modules in Python 3, and then compile those down to Python 2 at which point they can talk to the existing Python 2 code freely.

> Unless your 3to2 would literally reimplement Python 3 in Python 2

That's exactly what I'm suggesting. Take the existing Python 3 compiler and swap out the backend to generate Python 2 code instead of bytecode.

This is stuff people do every day for other languages. It is very far from rocket science.

Re:Node Modules at War: Why CommonJS and ES Modules Can’t Ge... by chrisweekly on Thursday August 06, 2020 @11:22AM (#24071733)
Amen. Babel gets a lot of things right (and arguably has a tougher job than a 3to2 compiler would, given the speed of change and the target breadth / problem surface area).
Re:Node Modules at War: Why CommonJS and ES Modules Can’t Ge... by masklinn on Thursday August 06, 2020 @11:43AM (#24071960)
> arguably has a tougher job than a 3to2 compiler would

It doesn't. Not that its job is easy, but its job is to take constructs which are brand new and translate them to an older version of the language.

Of the same language.

It doesn't have to deal with the language or the APIs literally working differently when used the same way.

The Python version of Babel would be 3to3 compiler.

Re:Node Modules at War: Why CommonJS and ES Modules Can’t Ge... by kqr on Thursday August 06, 2020 @12:43PM (#24072678)
Babel is not limited to JavaScript frontends. You can take any language and write a Babel front-end for it. This is, for example, how Fable compiles F# code to JavaScript. Those two languages are very different from each other -- far more differences than Python 2 and Python 3 -- and it works brilliantly.