I can go on, but it's a terrible hodge-podge of systems. It works nicely for simple cases (consuming libraries off Github), but it's awful when you go into details. And it's not even used by its creators - since Google has a monorepo and they actually use their internal universal build tool to just compile everything from source.
The flip side of this is that it never has to worry about naming collisions or namespacing: Your public package name must be a URL you control.
Additionally, there is no requirement for a centralized package facility to be run. The Golang project is currently running pkg.go.dev, but that's only been in the last few years; and if they decided to get rid of it, it wouldn't significantly impact the development environment.
Finally, the current system makes "typo-squatting attacks" harder to do. Consider the popular golang package github.com/mattn/go-sqlite3. The only way to "typosquat" the package is to typosquat somewhere up the dependency tree; e.g., by creating github.com/matn/go-sqlite3 or something. You can't typosquat github.com/mattn/sqlite3, or github.com/mattn/go-sqlite, because you don't own those namespaces; whereas with non-DNS-based package systems, the package would be called `go-sqlite3`, and `sqlite3` or `go-sqlite` would be much easier to typosquat.
All those things I find really valuable; and honestly it's something I wish the Rust ecosystem had picked up.
> It requires users of your code to update all of their import statements throughout their code whenever you move your hosting.
This is a necessary cost of the item above. It can be somewhat annoying, but I believe this can be done with a one-line change to the go.mod. I'd much rather occasionally deal with this.
> It requires you to physically move all code to a new folder in your version control if you want to increase the major version number.
And the benefit of this is that legacy code will continue to compile into the future. I do tend to find this annoying, but it was explicit trade-off that was decided back when they were developing their packaging system.
Packaging is a hard problem, with lots of trade-offs; I think Go has done a pretty good job.
One way in which Go and Rust have it easier than Python or Node is that the former only have to deal with developers; the latter have to deal with both developers and users, whose requirements are often at odds with one another.
Non-DNS-based packages don't have to be named "go-sqlite". You can easily require some namespacing, and even use DNS as a base for that, but having an abstraction over it that recognizes the specific needs of package management is better. For example, Maven packages are called things like org.apache.commons, and registering a new package requires control of an equivalent DNS domain. However, if you later lose control of that domain, the new owners don't simply get to replace the packet in Maven just because they snipes your domain.
Go's choice to require full paths for import in each file is also not a direct implication of the previous item - they could have allowed the go.mod file to specify the path and a name, and then allow source files to import based on that name. Instead, this comes from Go tooling that existed before modules support, when the tooling would scour all files in your project to find dependencies.
Moving code to a v2 dir does not specifically help (or hinder) with backwards compatibility. Old code can always simply keep using the old versions of the package anyway. It is also a very unpopular decision, with very few packages actually adopting v2 versions precisely because of this requirement, even when making major breaking changes. Even the team maintaing the Go protobuf bindings decided not to use v2 when they rehauled their code (opting instead to create a new v1 at a new location with minor versions starting at 1.20...).
Sure, packaging is hard, but the Go team has chosen to go against the flow of essentially all other package managers, and instead of learning from their mistakes, they seem to have decided to make original mistakes all their own.
This removes a whole slew of attack surfaces that JS and Python have, and leverages an existing “prove you are who you say you are” infra
The advantage being, if you later lose access to the DNS domain that you used to publish an artifact to Maven Central, the new owner doesn't automatically get to compromise your artifact for all (new) users.
> It requires you to physically move all code to a new folder in your version control if you want to increase the major version number.
This is untrue.
> It requires users of your code to update all of their import statements throughout their code whenever you move your hosting.
This is only true if not using a vanity URL, but is sadly often the case.
> It takes arcane magic to support multiple Go modules in the same repo.
I don’t know what you’re calling arcane magic here, but we maintain repos at work with 6-7 go modules in without it being an issue whatsoever, and no “arcane magic” required, so I’m going to go ahead and say this is untrue too.
[1] https://stackoverflow.com/questions/10687627/relative-import... [2] https://stackoverflow.com/questions/57806081/how-to-access-n...
I was indeed wrong, this is not necessary, though it is the original strong recommendation of how to do it from the original modules proposal.
Still, slightly changing the critique to say that changing major version is a big hassle, and it requires touching all of your project files, and pointing out how bizarre the official recommendation is compared to other packaging systems keeps my point intact, I believe.
> This is only true if not using a vanity URL, but is sadly often the case.
Yes, I am aware that you can buy a custom DNS and point it to your repo to release under a better name, but it is almost never done (I think the only dependency the project I work on has that does this are Google's Go protobuf bindings and several k8s.io projects).
> I don’t know what you’re calling arcane magic here, but we maintain repos at work with 6-7 go modules in without it being an issue whatsoever, and no “arcane magic” required, so I’m going to go ahead and say this is untrue too.
If you want to maintain multiple Go modules in the same git repo, and you want others to be able to download them, you need to tag commits for each module you are releasing. These tags must be formatted to match the dir path to the specific module's go.mod file (which also "conveniently" becomes part of the module name), except for the afore-mentionet vX directories if you chose to follow the Go team's recommendations for major versions. Then, for local development, you also need each go.mod to contain a REPLACE directive for each other module inside the same repo (or maybe directives in a go.work file, or who knows what else). Overall, this ends up creating lots and lots of useless tags, and is the definition of what I'd call arcane.
You'll not find the tag format in any "getting started" doc, and you'll not get any help from go mod itself if you get any of this wrong - just some "not found" style errors.
Edit: oh, and I should note: I have no idea how this is supposed to be done if your repo is not in Git, even though go modules are supposed to support other VCSs too.
That's simply not true. That's only one way you can do it. Another way is to create a branch.
That was my impression some time ago.
But last week I attempted to compile a couple of (not very big) tools from cargo. And it ended up downloading hundreds of dependencies and gigabytes of packages.
Looks like node_modules.jpg all over again :(
Further, another major difference is that you don't need those dependencies after you've built. You can blow them away. Doing that with node is not as straightforward, and in many cases, not possible.
I wrote a post highlighting Go's mod system: https://verdverm.com/go-mods/
imo, it is the best designed dependency system I know of. One of the nice things is that Go uses a shared module cache so there is only one copy on your computer when multiple projects use the same dependency@version
The best solution always lies somewhere in the middle. But finding this “middle” (and adhering to this approach) is the hard part.
I don't see it being talked about here, either. Our current system of pulling in packages made by random people on the internet is going to burn us. We assume that everyone who creates a package is an honest, reliable, developer who will not inject malicious code into their package. This assumption is similar to the assumptions we made around SMTP, HTTP, DNS, and every other internet protocol. Turns out we were wrong and surprise! you can't trust people on the internet.
I'm not sure if we can solve this with package managers. But package managers are part of the culture that has created this problem, and are probably a reasonable starting point to try and address it.
[0]:https://david-gilbertson.medium.com/im-harvesting-credit-car...
The rest of the stuff around modules, like crippled constraints, zero control over contents (which they have changed!), and completely non-existent "x is available, upgrade" or release tooling: constant, unnecessary pain, and it'll be inflicting serious damage on the ecosystem for many years to come.
Do you mean ranges on dep versions? The way it is currently, the version you set is the minimum, and the algo finds the highest minimum set across all deps and uses that.
If ranges were introduced, you'd end up with an NP hard problem and need a SAT solver for your deps again
> Release tooling
What are you looking for here? Libraries only need to push a git tag, binaries do require a bit of work, but Goreleaser fills that pretty nicely. It would seem hard to standard where binaries would be pushed
> completely non-existent "x is available, upgrade"
https://go.dev/doc/modules/managing-dependencies#discovering...
"go list -m -u all"
Go's algorithm is much simpler and does not need a lockfile while still giving deterministic results. Ranged deps without a lockfile cannot. There is benefit to two people running the same command and getting the same dependency versions. Most projects do not start with a lockfile, so it is quite easy to have different versions when running getting started commands.
Why choose SAT over simpler deterministic algos?
Version incrementing, packaging wasms, dancing around code generation – all doable, but not standardized.
There's a release-please to automate all that, but it's not an easy task to set it up in all of your repos.
Besides, if in addition to Rust projects, you have projects in other languages like JavaScript, then you have to do it twice and struggle with understanding all of the package management systems provided by all languages you have.
A single swiss-army-knife package manager would be amazing.
We should have left Commonjs a long time ago, while keeping backwards compatibility.
At the same time what I see with the node+npm system is that everything is just ,,it just doesn't work by default''.
Having 10 other package managers doesn't work either, they are faster, but don't solve this problem.
This one issue (IMO) is preventing parts of the ecosystem from switching to esm.
Plus none of them handle binary library distribution as some of the packing models that came before them.
[1] https://go.dev/blog/migrating-to-go-modules current system
[2] https://news.ycombinator.com/item?id=34310674 goproxy agressively polling sourcehut
It's worth recognizing that go isn't alone in dns-based namespacing : java's maven/gradle use the same strategy: https://repo1.maven.org/maven2/gov/nih/imagej/imagej/1.47/
And given that packages have to depend on other packages, and cannot depend on a git repository, that feature is mostly useful for testing bug fixes, private repos for leaf packages, stuff like that.
That the entire code of all dependencies is hosted in one place is an important differentiator of crates.io vs what Go does, which is why this is relevant in this context. Crates.io is centralized, with all of the advantages and disadvantages that that brings, and Go is decentralized, with all of the advantages and disadvantages that brings.
My experience is that the older gen languages you mention had to invent package management, made lots of understandable mistakes and now are in a backwards compat hellscape.
Rust and Go built their packaging story with the benefit of lessons learned from those other systems, and in my experience the difference is night and day.