It's a mystery how DockerHub remains free. The network and storage costs must be massive. Docker also sold it's enterprise business, so I am not sure who is paying for all this.

If you're running a package registry, you'd be painfully unaware of your own requirements if you're using a host/cloud that you need to pay for the amount of bandwidth.

Hosting a package registry on AWS for example, unless you figure out a way of seriously reducing the amount of traffic (which seems to be against working towards making a popular registry), is suicidal because of the bandwidth costs.

This massively depends on the architecture of the package manager though. Composer (PHPs primary package manager) only serves the metadata and directs the installer to the already packaged ZIPs hosted on Github/Bitbucket/Gitlab. This allowed them to serve the whole community as a small team of 2-4 for almost a decade now without any outside funding.

> This massively depends on the architecture of the package manager though. Composer (PHPs primary package manager) only serves the metadata

See also, CocoaPods. But we're really talking about two things here as if they're the same, and I think it's because the language we use for them isn't clear: package managers (like CocoaPods and Composer) and full-blown package registries (like RubyGems and NPM). I mean, the difference between them is slight; and there's overlap. They both resolve package dependencies and sort out problems, for example. But they're such very, very different things!

It's not really fair to compare these two approaches and say things about Composer like "this allows them to serve the whole community with a team of 2-4 people": they're just not directly comparable. Those 2-4 people + the entire GitHub staff is what supports the whole community.

Now - don't get me wrong: I think it's fine that GitHub does this! I'm not saying that CocoaPods and Composer are making the wrong choice here at all. In fact, I'm proud to work for a company that provides this service to everyone. But it hand-waves away a lot of stuff as just an "architecture" decision, when in reality it's a decision not to do the hardest part - which is hosting, because it requires gobs of money and dedicated staff and people on-call 24/7.

Framing it as just a different "architecture" implies that registries like NPM are making the wrong choices by hosting their own downloads, which I think is not nearly that simple. Because what happens if GitHub or Bitbucket decided to not allow this type of usage? It would be devastating for the community - so I think it's very appropriate for package managers to make a decision about whether or not they also want to be a registry. They're considering what could happen down the road and what that would mean.

CJ's comments on all of this, re: entropic, are really good: https://www.youtube.com/watch?v=MO8hZlgK5zc

Composer is amazing and I'm going to say far better than what I've seen in JavaScript and python world.

Speaking of... The NuGet registry in GitLab deserves an option to run without authentication from other builds and/or the "public" accessing it. Currently, it does not even handle the build job token and I have to cross connect builds using my personal access token (which then again is accessible to other maintainers of a repo).

GitLab PM here - Agreed that is an important feature and one we have scheduled for milestone 13.1 which will be released in June 2020.

As a workaround, you can add your access token as an environment variable and use that to publish/install packages via CI/CD.

And if you are interested in contributing to GitLab that issue is a great way get started.

The issue for this is at https://gitlab.com/gitlab-org/gitlab/issues/36424

That is what I did. However the secret is then available to all admins. Anyway. Good to know it is scheduled.

We stopped using gitlabs artifact repositories for this exact reason!

We'd love for you to try again. We do support using the job token with our NPM, Maven and Conan repositories. What do you currently use?

Why is non canonical better? Because they get to charge a fee?

Yep, non-canonical registries are used by business and they are more likely to pay. For example our dependency firewall will be a paid feature that delays updates from packages that are recently updated under suspicious circumstances like: author changed, author information changed, different programming language, activity after a long period of inactivity, large change to the code, etc.

> delays updates from packages that are recently updated under suspicious circumstances

This is an example of the security curation you note earlier. People reviewing flags alongside the code, remediation during the delay, highlighting package authors you like are some of why “Being the canonical registry … tends to be a huge expense.” How many people build and run GitLab Package Registry?

We don’t know exactly how many people use it since it is open source but I think between 100,000 and 1 million people. The docker container registry is the most popular.

The security curation examples I mentioned are intended to use automated signals so there wouldn’t be an expense for labor. In fact we intend to run them separately on every self managed GitLab installation.

I am curious what proportion of compromised packages fall under the umbrella of scenarios you have outlined.

Me too. If someone has data on predictors of compromised packages I’m all ears.

Yep, in particular the use case is private packages.

There's also https://packagist.com/. I don't know how well the business is doing.

Can anybody comment on how hosting a package repo for someone large may help peering? I'd imagine they would see significant upstream traffic and globally but not certain if that makes an impact

I think hosting a package repo will generally create more downloads (costs money) than uploads (helps with peering but costs storage) because the average number of downloads for a package is more than 1.