Fundamentally the problems are the same, but the communities have very different priorities. The vast majority of Node is still web dev and the community focuses on dependencies that are deployed as JavaScript files (including transpiled code); this allows NPM to mostly ignore issues related to linking to native binaries and put more focus on those user-facing features you see and like. Python on the other hand has a lot of stakes on interacting with native binaries (scientific computing, automation, etc.) and with systems older than Node’s existence. This consumes up a ton of the community’s resource, and every new UX improvement also needs to take account of those use cases. If you’re ever tasked with keeping projects with native dependencies working over a moderate period of time in both languages, you’d gain a lot of respect on how well Python packaging works.

I don't know how common this is, but some NPM packages involve native binaries, so it must be doable. pg-native for example.

pg-native is a good example actually. Its readme lines out how you need to first get a compiler, libpq, and have some certain commands in your PATH. With psycopg2 (Python’s equivalent), the most common scenario is ‘pip install paycopg2-binary’ and you’re good to go.

So the difference is psycopg2-binary* bundles the libpq native code and pg-native doesn't? I'm no expert, but I think npm packages can include native code if they want thanks to node-gyp, only the node-libpq (which pg-native relies on) author seemingly decided not to package in libpq itself.

* Back when I used this, there was just psycopg2 which had the bin included.

Correct, but the challenges are to compile the dynamic library correctly so it runs on another machine, and on a given target machine choose to load a correct compiled artifact. Python has extremely good support for those compared to other similar ecosystems. The fact that most Python packages decide to do bundling on a comparably broard set of platforms, while packages on some other language are not, is a window to understand how the operation is made much easier by the ecosystem.

> How is [Python] very different from NodeJS?

Unlike Node, Python is essentially older than modern package management. When Python developers first decided to tackle distributing their code, `apt-get` did not yet exist.

Early approaches which stuck around way too long let any package do more or less anything at install time, and didn't bother with static metadata (can't figure out what your deps are except by attempting an install!). Subsequent solutions have struggled to build consensus in an already vast and mature ecosystem. Backwards compatibility means compatibility with bad conventions.

Not really. CPAN was already a thing when Python itself was released, long before there was any Python packages to share.

* * *

That makes sense. Backwards compatibility can be a large burden. I'm still sad that they haven't managed to make a new thing that just works. NodeJS is repurposing a language and runtime originally meant for hacky web scripting, but it ended up ok in the end.

If they don't fix their issues, someone else will provide a solution at a higher layer. Now it's common to install a Docker image to run something instead of a few Python deps.