Nothing ever works like a sane person would expect. When I do get a working typescript + webpack setup it’s almost entirely by accident. I stumbled upon something that works? Just avoid touching it at all costs.
Recently I wanted to load an AMD module in Deno and just about gave my self an aneurism before giving up hours later.
I have worked in dozens of languages over my career, and not encountered as much of a clusterf*ck as JavaScript anywhere else. I have never worked with anything before where so few people had any idea how it actually worked and just prayed that it continues.
I've written JS professionally for probably 12 or 13 out of the last 22 years, but I skipped the first couple iterations of frontend bundlers and tooling (kept rocking it old-school and just including plain .js files with script tags) until React, so I seem to have missed that intermediate phase, and now I cannot find it. I've lost probably 3 to 4 hours to trying to figure this out, last couple times I looked.
npx esbuild index.js --bundle --sourcemap --minify --outfile=out.js --format=esm
npx should download and run the binary for you and npx is almost always installed with node automatically.
When I have to use build-tools I usually go for esbuild.
Uh what? It's literally the name itself "node"_modules, it's designed to be used with node. The fact that it use same language and can later be used by browser is another matter. Thankfully we can easily bundle the whole thing using webpack nowadays.
There are also countless boilerplate project with webpack setup already so you can get reference from there.
That’s the problem. It’s not JavaScript. It’s not any part of the ECMA standard, it doesn’t work in runtimes other than node (read: the browser) yet it lives in the .js files making them not truly JavaScript. You have split the language.
The syntax isn’t part of the language and was designed in such a way that it cannot be polyfilled. This lead to a thousand packagers that would not be necessary at all if a sane syntax had been chosen. The creator of node himself has said making node incompatible with the browser was one of his biggest mistakes.
https://medium.com/@imior/10-things-i-regret-about-node-js-r...
> There are also countless boilerplate project with webpack setup already so you can get reference from there.
That’s literally the other problem! People just copying boilerplate without understanding it. That’s not a proper solution. That’s not a maintainable project, that’s copying other peoples homework and getting yourself in over your head.
Well I can understand it. As I've used the previous generation of jQuery, knockout, etc, nodejs and it's tooling makes things easier to scale, even with bigger overhead cost.
EDIT:
> That’s literally the other problem! People just copying boilerplate without understanding it. That’s not a proper solution. That’s not a maintainable project, that’s copying other peoples homework and getting yourself in over your head.
Copy, make it run well, modify, learn is how I learn things. Without working example, it's magnitude times harder to learn something. The fact that people copy and use without trying to understand things is their fault, not the hundreds of boilerplate code.
Most of my code today is very straightforward, start a project, add type: module and 90% of the packages will work. If it doesn't, I either pick one of the 1M+ alternatives, try to decipher the issue (often is changing the import format or entrypoint), or avoid using a package for the problem I was trying to solve if trivial enough.
Node modules are just a reaction to the actual original sin, which is building a popular language that essentially had no standard library. I suppose that's the kind of thing that will happen when you have to build one in a matter of weeks, but everything from webpack to left_pad is essentially the fault of the language not having a standard library.
Moving javascript out of the browser and trying to use it as a one-size-fits-all replacement for everything from systems to application programming, and every language from C++ to Java, is the original sin. Stop complaining because Brendan Eich's toy scripting language isn't a good replacement for things it was never meant to be used for. A screwdriver isn't a good hammer, either.
I have no particular love for using JS as a language to power everything from IoT devices to web servers, but it doesn't really matter how I feel about it. It's not going anywhere.
Why should the language provide that? What other languages provide an equivalent to that? left_pad is intended for formatting text within a terminal, not a website. There's no need for it within the context of a website, because HTML has tables, CSS and other elements for that.
https://www.php.net/manual/en/function.str-pad.php
https://docs.ruby-lang.org/en/3.1/String.html
https://developer.apple.com/documentation/foundation/nsstrin...
> left_pad is intended for formatting text within a terminal, not a website. There's no need for it within the context of a website
This is just not correct. It’s for padding a string with any character, for example adding leading zeroes to a number for display purposes. There are plenty of reasons to want to do that in a browser.
My code is a combo of lodash bits, datefns, helper regexes I’ve curated over the years, etc
Is there a standard library-like package that you import on most projects?
It's inevitable that the conclusion will be the same if the same steps are taken in the same direction. The counter-arguments fail to recognise the special case of "most of us are from Mozilla and we all get along[1]", "no-one has attacked us yet so no-one ever will[2]", "it's tracktable to untangle the dependency hell in your head because there aren't that many crates (yet)", and of course "we all know this history like the back of our hand, because we've all been around since v0.1 alpha."
I'm picturing "10 years later" when some junior developer at MegaCorp Inc desperately tries to figure out why the legacy Rust 2018 version he's forced to use in 2032 can't compile some privately hosted crate that's needed for an enterprise project. It'll probably depend on a specific version of a crate that was yanked because some transitive dependency had to be yanked because that guy was a Russian kid that sold access to this popular crate for $500 to Israeli spies. Of course now the patched crate requires Rust 2030 or whatever, breaking fifty seven of the two thousand transitive crates being pulled it, so good luck to him with disentangling that mess and making this work.
PS: I wish I was making this story up, but I basically just changed Angular to Rust for a real thing that happened to me, like... a month ago. There is no fundamental difference to how Rust does packages, it's just younger so the mess hasn't accumulated to Angular's levels of madness. Give it time...
[1] Except for that spat with the core team.
[2] That we know of. Ignore the guy spamming popular, single-word crate names and shovelling in 99.9% C/C++ code with the thinnest possible Rust wrapper. Surprise! Your code is now more C++ than Rust.
Editions are not lock-in; the entire point of the edition system is to allow the version of Rust that exists in 2032 to compile every prior edition in the same project at once, and only single crates at a time have to be 2030 edition.
Before making any change that could possibly be breaking, the change is tested on every single crate on crates.io, as well as on a great many GitHub repos that aren't, in order to make sure there aren't any regressions (referred to as a crater run).
You are right that the problems you describe would be problems if Rust was implemented the way you're implying. That is why Rust was not implemented the way you're implying, but rather implemented specifically to avoid these problems.
The way you describe yanking also seems a bit confusing. If the original project had a dependency that was yanked for security concerns, then the original project was already compromised in the first place. Even then, assuming the lockfile was checked in, then it should still be possible to download the yanked file (plus the compromising code, although the act of downloading a yanked file produces a warning, I believe). If you mean that the project was working fine (no compromise) and then a later version of the dependency was compromised, then no changes are needed at all - the lockfile will always ensure the exact same version is downloaded.
If you genuinely think you've found a situation that isn't solved by the edition mechanism, or by the judicious use of lockfiles for all projects, then I'm sure the Rust team would love to hear it (although I can't speak for them). But from what it sounds like, you're describing issues with the Node ecosystem and assuming they also apply to Rust, despite a number of differences in how the two ecosystems operate.
There’s no downside beyond “it requires development time and maintenance” like any other feature.
I don't have a single, definitive, clear solution -- as pointed out by others -- nobody does. It's not a simple problem.
That doesn't mean that steps can't be taken to improve the situation, perhaps dramatically in some cases.
1) Enforced MFA to publish a crate -- credential theft is semi-regularly seen as an attack vector.
2) Strong links between the "source ref" and the specific crate versions. An example of this done super badly is NuGet. All of the hundreds (thousands?) of Microsoft ASP.NET packages point to the same top-level asp.net or .net framework URLs. E.g.:
https://www.nuget.org/packages/Microsoft.Extensions.Configur...
Links to "https://dot.net" as the Project Website, and "https://github.com/dotnet/runtime" as the repository. This couldn't be more useless. Where is the Git hash for the specific change that "7.0.0-preview.4.22229.4" of this library represents? Who knows...
3) Namespaces. They're literally just folders. If you can't code this, don't run a huge public website. This is more important than it sounds, because wildly unrelated codebases might have very similar names, and it's all too easy to accidentally drag in entire "ecosystems" of packages. Think of the Apache Project. It's fine and all if you've "bought in" to the Apache way of doing... everything. But imagine accidentally importing some Google thing, some Netflix thing, some Apache thing, and some Microsoft thing into the same project. Now your 2 KLOC EXE is 574 megabytes and requires 'cc', 'python', and 'pwsh' to build. Awesome.
For example, in ASP.NET projects I avoid anything not officially published by Microsoft and with at least 10M downloads because otherwise it's guaranteed to be a disaster in 5-10 years. Ecosystems diverge, wildly, and no single programmer or even a small group could possibly stitch them back together again. Either it's a dead end of no further upgrades, or rip & replace an entire stack of deeply integrated things.
4) Publisher-verified crate metadata / tags. You just cannot rely on the authors to be honest. It's not even about attacks, it's also about consistency and quality. All crates should be compiled by the hosting provider in isolated docker containers or VMs using a special "instrumented build" flag. Every transitive dependency should be indexed. Platform compatibility should be verified. Toolchain version compatibility should be established for the both the min and max range. Flags like "no-std" or whatever should be automatically checked. CPU and platform compatibility would also be very helpful for a lot of users. The most important one in the Rust world would be the "No unsafe code" tag.
This would stop "soft attacks" such as the guy spamming C++ libraries as Rust crates. Every such crate should have been automatically labelled as: "Requires CC" and "Less than 10% Rust code".
Similarly, if a crate/package changes its public signature in a breaking way, then the publishing system should enforce the right type of semantic versioning bump.
Essentially, what I would like to see is something more akin to a monorepo, but not technically a single repository. That is, a bunch of independent developers doing their own thing, but with a cloud-hosted central set of tooling that helps gain the same benefits as a monorepo.
I'm expecting a lot of arguments along the lines of "that sounds like a lot of work, etc..." Meanwhile Mozilla had a large team for this, millions of dollars of funding, and did not do even 0.1% of what Matt Godbolt did in his spare time...
Or even better, make crates request permissions for what kind of functions they can call, similar to the chrome plugin API. A graph crate doesn't need encryption, file opening or netstack permissions.
Someone could always roll their own IO, but self reporting and automated detection tooling (to catch those that slip through the cracks) would bring this percentage way down.
Maybe the language could even evolve "unsafe" for IO, even if just as a flag for users. That way it would all be incredibly easy to audit.
Refactoring breaks everything - just look at Python 2 to Python 3. Part of the problem is having a bill of materials and having accurate tracing of the ingredients of a build, such as reproducible builds. But the time investment in these is hard work.
Computers are good at cross referencing. If you indexed everything in a graph database. It should be a simple graph Search to find dependencies of a git commit hash, binary sha256 installer file. And a web of sha256 relationships. And mapping between them and commit shas. It would be useful for security too but also interested in things being robust and reliable. Like you say it's a monorepository but not one.
Tooling is what's needed
The python packaging experience and ruby bundler and npm and other version manager experiences simply leads to common breakage.
A common following-up question the Haskell people used to say when that was being discussed there is, if your build system can do that, why do you need semantic versioning for?
Still, it's better than doing nothing. There's nothing similar to Stackage or Backpack on Rust, so it would be a clear gain. It's just that you can go further.
In my neck of the woods, Python is the major source of time spent trying to get other people's code to run on my machine. The path of least resistance for Python developers tends to produce code that requires you to go through an easter egg hunt to make it run outside the original author's machine.
Yes, you could write Python utilities that are easy to install and run, but people don't. And the last bit of that sentence is the one that actually counts. "Could have" doesn't actually count in an engineering context. If the consumer has to assemble and repair software before getting it to run, the developer isn't being very professional.
Specifically, I think Apple's philosophy of "everything comes with its own dependencies embedded" saved us a lot of trouble. Sure, it's more disk space and potentially lots of duplicates (just count the number of times various SSL libs are included in a single large and messy application), but it usually just works.
Though it's still not great if everyone uses their own SSL library, because then you can't update it if there's a bug.
Javascript is deployed in many different environments and you need to learn not just how to include a library in each environment, but also how to interact with that library. Also unlike C or other languages - Javascript has a hard time warning you about when you've imported something the wrong way. C can at least tell you "this does not link" (so it was found but is binary incompatible) - in my experience if you copy and paste a node import line in the browser the browser will just say "this does not make sense to me." C gives more breadcrumbs.
I also think that once you get outside the C-likes, the story is universally better. For all of its packaging flaws, it's easy to get python packages for development.
Every time I want to make something in Javascript I'm frustrated all over again by how I need to know details about the Javascript ecosystem that I have no way of knowing at this stage (like: getting to hello world can require a working knowledge of WebPack). It's just no where near that bad other places.
I do not, but again - I think "I should be able to run any version of a language in any place" is not related to "it is strangely hard to run a given Javascript file."
I actually think Lua is a great example for comparison that illustrates the point! Different Luas support different features, but the basics (import statements, etc) are syntactically valid (and singular) in all of them. If you try to import a library that doesn't exist in the version you've embedded, you will get an import error. In Javascript - there are multiple import systems for different ecosystems. Getting an import error might mean you need to install the library, it might mean you're using the wrong convention, it might mean that the version of the library has a compilation error....etc. Javascript does not really help you resolve this.
What this means is that there's a clean separation between environment and language. If your Lua deploy supports all the features your script uses, you are good - there are no "families" of Lua script that use different conventions for importing libraries or interop or whatever. A given piece of Javascript, however, has an arbitrary number of deployment-specific setup actions that require pretty advanced ecosystem knowledge to identify.
Like, look back at the errors listed in the article. There's nothing "wrong" with the restrictions, but the language ecosystem does an unusually bad job of helping you understand and navigate them. Like, you can have two or three different import mechanisms for different environments...but if you want to do that you should be more helpful in pointing out when you're using the wrong one.
console.log("Hello, World")
Meanwhile, if you're interested in using someone else's code, the CDNs are all still there. Vue, JQuery, lodash, and many other libraries are available for import directly into your HTML file.
A lot of people in this thread are conflating "all the tutorials tell beginners to do things the hard way" with "JavaScript is now a harder language to get started with than it once was".
It's not weird for a Java project to include a library that hasn't had any updates in 4+ years. Sometimes those sorts of libraries are even considered the standard for whatever thing they do.
Meanwhile in JS you have to assume a library with no releases in the last 6 months has already bitrotted to death.
It just worked.
And code was quite readable. I mean it not as a complement to myself, rather the fact that Go is designed to be readable and it pays off.
I only had to add modules support (basically run `go mod init` in the library's folder) to make it 100% modern-version friendly.
That was as close as I could be to successfully complete 10 Years Reproducibility Challenge [1].
I don't expect to walk in to a workshop and find the welding tools intuitive to use with no training, and I don't expect npm modules to be intuitive to an outsider either.
Don’t look at how things are sold, look at how they are. In this case it’s code, of course there’s complexity.
Young languages are easy to get started in. Ruby used to be.
Though ruby has a separate problem of breaking changes. The back-compatibility crowd can run old code (e.g. java), but at the cost of their soul.
Rust is a new language, and it has the same complaints above.
edit: After reflection, I think Python straddles a middle ground. If you use modern packages and Python 3, it can be a smooth experience. If you use older (improperly-specified) packages, or FFI-wrapped-libs, it can turn into a mess too.
I just want to try this C++, download, unzip, oh it's windows so .project file. Fine, redo on windows , oh it's 3 versions of vstuido old and says it wants to upgrade , okay. Hmm errors. Try to fix. Now it's getting linking error.
repeat the same with xcode and any project in c, c++, objc, swift
okay how about ruby? oh I have old ruby , hmmm , try to install new ruby, seems to run, but it can't find certain gems or something. oh and this other ruby thing I was using is now broken ? why do I have to install this stuff globally? You don't but there are several magic spells you must execute and runes you must set in the rigtt places. Oh ... I think I got that setup but it getting this new error.
....
I'm sypathetic it sucks but it's not unique to js