News for hackers. Stuff that matters.
Re:Ask HN: Why does every package+module system become a Rub... by tsimionescu on Monday January 30, 2023 @06:21AM (#34578352)
Go package management is a mess of hacks. It doesn't even have a repository, instead relying on source control systems to do the actual work, with special hacks for each source control system to define the artifacts that can be downloaded (e.g. using Git tags with some magic format, or Perforce commit Metadata). It requires you to physically move all code to a new folder in your version control if you want to increase the major version number. It requires users of your code to update all of their import statements throughout their code whenever you move your hosting. It relies on DNS for package identity. It takes arcane magic to support multiple Go modules in the same repo.

I can go on, but it's a terrible hodge-podge of systems. It works nicely for simple cases (consuming libraries off Github), but it's awful when you go into details. And it's not even used by its creators - since Google has a monorepo and they actually use their internal universal build tool to just compile everything from source.

Re:Ask HN: Why does every package+module system become a Rub... by gwd on Monday January 30, 2023 @07:59AM (#34579020)
> It relies on DNS for package identity.

The flip side of this is that it never has to worry about naming collisions or namespacing: Your public package name must be a URL you control.

Additionally, there is no requirement for a centralized package facility to be run. The Golang project is currently running pkg.go.dev, but that's only been in the last few years; and if they decided to get rid of it, it wouldn't significantly impact the development environment.

Finally, the current system makes "typo-squatting attacks" harder to do. Consider the popular golang package github.com/mattn/go-sqlite3. The only way to "typosquat" the package is to typosquat somewhere up the dependency tree; e.g., by creating github.com/matn/go-sqlite3 or something. You can't typosquat github.com/mattn/sqlite3, or github.com/mattn/go-sqlite, because you don't own those namespaces; whereas with non-DNS-based package systems, the package would be called `go-sqlite3`, and `sqlite3` or `go-sqlite` would be much easier to typosquat.

All those things I find really valuable; and honestly it's something I wish the Rust ecosystem had picked up.

> It requires users of your code to update all of their import statements throughout their code whenever you move your hosting.

This is a necessary cost of the item above. It can be somewhat annoying, but I believe this can be done with a one-line change to the go.mod. I'd much rather occasionally deal with this.

> It requires you to physically move all code to a new folder in your version control if you want to increase the major version number.

And the benefit of this is that legacy code will continue to compile into the future. I do tend to find this annoying, but it was explicit trade-off that was decided back when they were developing their packaging system.

Packaging is a hard problem, with lots of trade-offs; I think Go has done a pretty good job.

One way in which Go and Rust have it easier than Python or Node is that the former only have to deal with developers; the latter have to deal with both developers and users, whose requirements are often at odds with one another.

Re:Ask HN: Why does every package+module system become a Rub... by tsimionescu on Monday January 30, 2023 @08:46AM (#34579480)
People lose DNS names by accident all the time. It's also easy to typosquat many DNS domains, and even Github projects occasionally.

Non-DNS-based packages don't have to be named "go-sqlite". You can easily require some namespacing, and even use DNS as a base for that, but having an abstraction over it that recognizes the specific needs of package management is better. For example, Maven packages are called things like org.apache.commons, and registering a new package requires control of an equivalent DNS domain. However, if you later lose control of that domain, the new owners don't simply get to replace the packet in Maven just because they snipes your domain.

Go's choice to require full paths for import in each file is also not a direct implication of the previous item - they could have allowed the go.mod file to specify the path and a name, and then allow source files to import based on that name. Instead, this comes from Go tooling that existed before modules support, when the tooling would scour all files in your project to find dependencies.

Moving code to a v2 dir does not specifically help (or hinder) with backwards compatibility. Old code can always simply keep using the old versions of the package anyway. It is also a very unpopular decision, with very few packages actually adopting v2 versions precisely because of this requirement, even when making major breaking changes. Even the team maintaing the Go protobuf bindings decided not to use v2 when they rehauled their code (opting instead to create a new v1 at a new location with minor versions starting at 1.20...).

Sure, packaging is hard, but the Go team has chosen to go against the flow of essentially all other package managers, and instead of learning from their mistakes, they seem to have decided to make original mistakes all their own.

Re:Ask HN: Why does every package+module system become a Rub... by jakewins on Monday January 30, 2023 @10:56AM (#34581393)
Java enforces domain name ownership as well; I can’t publish to maven central under io.netty.

This removes a whole slew of attack surfaces that JS and Python have, and leverages an existing “prove you are who you say you are” infra

Re:Ask HN: Why does every package+module system become a Rub... by tsimionescu on Tuesday January 31, 2023 @12:58PM (#34598857)
Note that Maven Central only imposes this once when creating a new artifact - which is, I believe, much better than Go, which imposes this for every go mod download.

The advantage being, if you later lose access to the DNS domain that you used to publish an artifact to Maven Central, the new owner doesn't automatically get to compromise your artifact for all (new) users.

Re:Ask HN: Why does every package+module system become a Rub... by jen20 on Monday January 30, 2023 @10:23AM (#34580822)
Some of your criticism is reasonable, and I’m no fan of Go’s module system as a standalone artifact, but much of your criticism is unfounded.

> It requires you to physically move all code to a new folder in your version control if you want to increase the major version number.

This is untrue.

> It requires users of your code to update all of their import statements throughout their code whenever you move your hosting.

This is only true if not using a vanity URL, but is sadly often the case.

> It takes arcane magic to support multiple Go modules in the same repo.

I don’t know what you’re calling arcane magic here, but we maintain repos at work with 6-7 go modules in without it being an issue whatsoever, and no “arcane magic” required, so I’m going to go ahead and say this is untrue too.

Re:Ask HN: Why does every package+module system become a Rub... by Cieric on Monday January 30, 2023 @12:12PM (#34582901)
Not the one you we're replying to, but I have an idea of the arcane magic he might be referring to. Back when I started learning Go I wanted to make a few example projects to test how well the language works, but I didn't want to push anything anywhere. When I tried to break my project into modules it ended up being that I needed to give them fake urls or something and then tell it to redirect the fake url to a folder instead. This[1] and this[2] gives a good example of the stuff I was running into at the time. I also realize that I should have probably been using sub-packages and not sub-modules, but coming from other languages I didn't realize they were suppose to be different. The thing that screwed with me most though was that there were half solutions for sub-modules to work so I kept using that terminology when searching google.

[1] https://stackoverflow.com/questions/10687627/relative-import... [2] https://stackoverflow.com/questions/57806081/how-to-access-n...

Re:Ask HN: Why does every package+module system become a Rub... by metaltyphoon on Monday January 30, 2023 @12:26PM (#34583158)
Now a days you can use go workspaces to do this so you don't have to modify your go.mod files anymore.
Re:Ask HN: Why does every package+module system become a Rub... by Cieric on Monday January 30, 2023 @01:08PM (#34583869)
Thanks for the info, I'll look into that the next time I pick up go again.
Re:Ask HN: Why does every package+module system become a Rub... by tsimionescu on Monday January 30, 2023 @12:47PM (#34583537)
> This is untrue. [needing to move code to a new folder to increase major version]

I was indeed wrong, this is not necessary, though it is the original strong recommendation of how to do it from the original modules proposal.

Still, slightly changing the critique to say that changing major version is a big hassle, and it requires touching all of your project files, and pointing out how bizarre the official recommendation is compared to other packaging systems keeps my point intact, I believe.

> This is only true if not using a vanity URL, but is sadly often the case.

Yes, I am aware that you can buy a custom DNS and point it to your repo to release under a better name, but it is almost never done (I think the only dependency the project I work on has that does this are Google's Go protobuf bindings and several k8s.io projects).

> I don’t know what you’re calling arcane magic here, but we maintain repos at work with 6-7 go modules in without it being an issue whatsoever, and no “arcane magic” required, so I’m going to go ahead and say this is untrue too.

If you want to maintain multiple Go modules in the same git repo, and you want others to be able to download them, you need to tag commits for each module you are releasing. These tags must be formatted to match the dir path to the specific module's go.mod file (which also "conveniently" becomes part of the module name), except for the afore-mentionet vX directories if you chose to follow the Go team's recommendations for major versions. Then, for local development, you also need each go.mod to contain a REPLACE directive for each other module inside the same repo (or maybe directives in a go.work file, or who knows what else). Overall, this ends up creating lots and lots of useless tags, and is the definition of what I'd call arcane.

You'll not find the tag format in any "getting started" doc, and you'll not get any help from go mod itself if you get any of this wrong - just some "not found" style errors.

Edit: oh, and I should note: I have no idea how this is supposed to be done if your repo is not in Git, even though go modules are supposed to support other VCSs too.

Re:Ask HN: Why does every package+module system become a Rub... by mseepgood on Monday January 30, 2023 @10:35AM (#34581017)
> It requires you to physically move all code to a new folder in your version control if you want to increase the major version number

That's simply not true. That's only one way you can do it. Another way is to create a branch.

Re:Ask HN: Why does every package+module system become a Rub... by tsimionescu on Monday January 30, 2023 @12:57PM (#34583689)
It's the recommended way, but you're right it's not the only one. Still, updating your major version number is much harder than in any other version control system I've seen (since however you do it, it requires you to update every single file in your repo to point to the new module). It also complicates the relationship between git tags, go.mod file location, and location within the Git repo significantly.
Re:Ask HN: Why does every package+module system become a Rub... by skovati on Monday January 30, 2023 @12:18PM (#34583012)
Interesting read on the official golang.org proxy DoSing git.sr.ht

https://sourcehut.org/blog/2023-01-09-gomodulemirror/