Amen to this. This type of stuff leads to so much confusion especially for beginners. I remember the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse when I started to get into front-end development. Yes having multiple choices lets us experiment with alternatives, but I think we underestimate the costs of complicating the ecosystem as a result.
Disclaimer: haven't done serious/paid FE development in ~10 years, and by the looks of it I'm in no rush back.
Oh man sounds like nothing changed. Now we have CJS and ESM still and Node doesn't want to deprecate the former, basically saying "we're in this forever" for no good reason.
I know one legitimate reason is "tree shaking" (source-level LTO when bundling modules). Dumber, static import/export statements probably simplify that in some way. Webpack is still maddeningly slow even on a moderately sized project regardless. Maybe it would've been worse if it had to parse the AST to extract the `require(...)` calls.
One change that ES modules introduced, I think, for no other reason than to be backwards incompatible, is changing the behavior of the default export (`export default foo` transpiles down to `module.exports.default = foo` instead of `module.exports = foo`).
Other "ohai guys this is the new normal now" kinds of changes are making the dynamic imports async-only (after not supporting them for a while) as well as changing the behavior of module resolution. Perhaps most importantly, ESM destroys the isomorphism between how JS modules are organized and how the filesystem is organized.
And the cherry on top is called TS-ESNode: https://github.com/K-FOSS/TS-ESNode because TypeScript modules and ESM are the same thing yet you need to somehow find this third-party shim which is required for them to work together at all. It's enabled by wrapping the interpreter, just like Yarn2's new dependency resolution.
npm install [package]
yarn add [package]
Even the options for this command are different (--save-dev vs --dev). In fact, the situation is even more confusing, since `yarn install` exists but does not accept parameters:
yarn install
[1/4] Resolving packages...
success Already up-to-date.
yarn install something
error `install` has been replaced with `add` to add new dependencies.
Run "yarn add something" instead.Why? Aren't there multiple package managers in many languages? SBT/Maven, Ant/Ivy/Maven, PIP/Poem(or poetry or whatever it is). What's the harm?
Perhaps I just don't have that much experience with pip, but it feels more like a tool to install packages, then you're on your own with some makefile or python scripts with no standard setup.
In most ecosystems, I can share package sources between a VM and a host.
npm shares mutable content (eg compiled artifacts) with package sources in `node_modules`. That breaks a ton of workflows that are common in other ecosystems.
For instance, in ruby apps using bundler, I can commit my dependencies in vendor/cache; there are no network dependencies other than fetching the source code. That makes turning code into a running server faster and more reliable.
Go supported the same thing from v1 via GOPATH (because that's how google runs their repo). Commit your dependencies and carry on (go had different shortcomings in dependency management in those days).
This feature turns things like `left-pad` from a fiasco into a non-event.
Also, it took npm years to implement a lockfile, only for most npm commands to disregard it. I used to frequently get versions other than the one I wanted. It's been ~6 years since I switched away so it may have improved since then. After my experience using npm I fundamentally do not trust the brand, and until yarn screws up I have no reason to give npm another chance.
In fact, both of these package managers have had direct support for this workflow since 2016 (https://docs.npmjs.com/cli/v7/using-npm/config#offline) and 2017 (https://classic.yarnpkg.com/blog/2016/11/24/offline-mirror/) respectively.
Experienced developers they are not.
Bundler, AFAIK, isn't first-party, its a separate team and project.
Historically, IIRC, gems wasn't even first-party
Good standard solutions are often the outcome of multiple competing efforts pushing forward, proving what works, and proving what doesn't.
But, it's irrelevant. I'm not arguing no one should make a third-party package manager, just that there should be a standard, ideally first-party one, that is good and well supported.
Languages like Ruby, Rust, Swift have recognized it is beneficial for the core project to provide a package management solution.
Which is to say, yes, it’s painfully half-baked and riddled with inconsistencies. But that is pretty typical of JavaScript anyhow.
Yeah but what I mean is that as it stands you literally cannot implement a package manager based on ES Modules alone. So it’s strange to me that someone would say that ES Modules are the package management solution for JavaScript. That’s not what it is currently for. Though the relevant standards may certainly be officially extended in the future, to allow for standardized package management based on ES Modules.
In principle, ES modules do away with needing a package manager. And yes, folks are already working on the consequential gaps in capability.
The point still being, however incomplete it currently is, this is the first-party solution. Node is not JavaScript.
At least one time, I tried to get into python packages, but I recall there seemed to be about a half a dozen methods for doing it at the time. I didn't try very hard.
I wish I could say I never had to think about it, though.
pip install pandas
??
The trouble is firstly that pip will happily install conflicting versions of numpy which breaks your code.
This occurs because pip didn't check dependencies until last year, which is one of the reasons it's a terrible package manager.
Now it just gets stuck spinning it's wheels for ages and then errors out, which is better but still not good.
Conda, while really slow actually handles c level dependencies which makes it usable.
So don't!
I consider it a public service to keep mentioning this though, so that in future jobs, I don't have to explain over and over again why I don't want to use pip to "manage" our dependencies.
The problem is not so much technical but rather social in my view. For one, it's quite "natural" for language developers to try to grow their community around its very own tools, even if those tools effectively "cut corners".
The other "social issue" has to do with practices: packagers and users of the generic GNU/Linux tools I mentioned have different expectations in terms of having a curated package set, ensuring common conventions are followed, building software from source, and so on. This is at odds with the practices encouraged by some of the language-specific package managers.
Npm does not try to manage complexity, as explained at https://dustycloud.org/blog/javascript-packaging-dystopia/, Java packages often come with opaque jars that nobody builds from source, sometimes due to circular dependencies, and so forth.
I chuckled a little when I realised that npm, Yarn, Yarn 2, and pnpm can all be found alive and well in the ecosystem
Deno is an interesting model, but maybe not widely applicable?
You might not need to transpile your app (via babel), but you still need to bundle it (via webpack/rollup/parcel/esbuild/swc/etc)
Unfortunately loading my app went from a second to half an hour (I think it just broke loading all the files somewhere).
But, also, you can do quite a bit with carefully selected small dependencies and a minimal amount of JS: I have some relatively complex sites that forgo bundling with no perceptible issues here
We try to write code that is isolated/modular for many non-controversial reasons. A common strategy is to co-locate related functions/classes/logic in one file and keep this separate from unrelated logic in other files. The more your code does, the more logic to build, the more files you'll have. The same holds for your dependencies. The more they do for you, the more files/modules they need to isolate their logic. Thus micro-dependencies have few files in two cases: 1) they do relatively little and shift responsibility to your code 2) they do a lot but have already combined/compiled their files into 1-2 files for you, often using a bundler.
ESM is an excellent native solution to referencing logic from other files so you can split & isolate your code when you write it. But this is an authoring strategy, and it should not dictate your distribution strategy.
I totally agree with you on pruning dependency trees though.
Many project are going to use some kind of central import/export system to keep track of external dependencies and that could perfectly well be the result of a package manager compiling the equivalent of a package.json to a .ts file.
Only-URL imports does not mean that no structure is available
Haven't used Bundler, but have you looked at Nix? It's got support for a bunch of languages/ecosystems, including the npm one: https://nixos.org/manual/nixpkgs/stable/#chap-language-suppo...
Either way, the trope doesn't really apply. Even if this were a new implementation, it wouldn't be a new standard. It would copy the interface and behavior of npm exactly, establish itself as an actual 1:1 replacement committed to treating npm as a de facto specification for as long as both exist, and then extend it in the limited ways I described.
Yarn made npm move in the right direction. Like in any other area, we need competition.
This also reminds me how much we keep reinventing the wheel on this. I know it's likely infeasible for various reasons but it seems like it would be great to have a language-agnostic package manager that worked more or less like Bundler and can solve these problems once.
(The most obvious downside of that is that – well – what language is it written in, and does it cause you to have to install multiple toolchains. Maybe the solution is some kind of standard spec for how package management should work, and the manifest formats, etc, and then each language ecosystem implements that standard. Or something. IDK.)
Edit: A cool feature of such a system could be installing dependencies from multiple language ecosystems via one manifest file.