For programming languages, there are several examples of commercially run package managers:
- the Java/Kotlin/Scala ecosystem is based around maven central, which is run by Sonatype, Inc.
- Go modules are hosted by Google. Previously, most libraries were hosted on Github
- Rust's crate index is on Github
- The Docker/Moby registry is run by Docker, Inc. (though that might be a stretch for "package manager" :))> Rust's crate index is on Github
Note that the crates.io index is just a single git repo that holds JSON metadata about each crate: https://github.com/rust-lang/crates.io-index . The actual code found on crates.io is hosted on S3. The index is an important part of the system, but there's nothing tying it to Github specifically.
Who's funding the S3 bill? Took a peek around crates.io but didn't see anything about it.
Ah, neat! Works out well for them; I assume it's relatively cheap to just serve an S3 bucket on their CDN (though I guess bandwidth costs may rise rather dramatically if Rust ever reaches Node levels of popularity), while not taking on any other operational expenses of actually running a registry.
CDN bandwidth by nature of the reach of Cloudfront will mostly be offload onto the local peering fabrics. It basically costs nothing in per mbit billing - e.g. linx https://www.linx.net/wp-content/uploads/2017/10/Fees_Schedul.... 100gb hand. That's not to say there aren't other major costs in running a resilent edge network which go someway to justify 0.0Y/GB pricing where Y varies from 1-9 depending on location for non sponsored projects. tl;dr - it won't ever be a problem
Please don't use code blocks for regular text and quotes. Really hard to read on mobile and narrow viewports.
Technically the Go module _proxy_ is hosted by Google. Even if the proxy went away, you'd still be able to get access to all of the packages as they're still hosted elsewhere. It just wouldn't be as fast.
Maven Central has mirrors and alternatives and you can trivially host your own repository, all you'd need is a plain web server serving a bunch of static files.
Some libraries aren't hosted on Maven Central actually, so it's not uncommon to see instructions for adding extra resolvers to your build config.
The Java ecosystem isn't as dependent on Maven Central as the JavaScript ecosystem is on npmjs.com
Re:NPM Is Joining GitHub
by brunoborges on Monday March 16, 2020 @05:16PM
(#22597940)
Almost every library out there is on Maven Central. Even Oracle JDBC drivers are now (finally) on Maven Central.
If MC goes away as it exists today, the Java Ecosystem will take a huge hit as almost every open source project would stop building in CICD environments from the get-go.
If it vanished instantly then yes, but a huge number of packages on Central are mirrors from jcenter. There are not only theoretical competitors to Maven Central but an actual widely used one (jcenter/bintray), which is easier to use anyway. There's also jitpack too. So people could migrate pretty quickly to alternatives.
You're not technically wrong, but I bet that 90% of the projects or at least the examples have some sort of intentional or unintentional reliance on Maven Central, that would break the build if it weren't there. Even a lot of companies that set up internal repositories don't realize that they hit still are gonna hit Maven Central initially, before everything is bootstrapped, depending on how things are configured. It would reflect just really poorly on the Maven ecosystem (or any ecosystem alike, I have to assume, but know less about) if the canonical repository would just... "poof".
It's actually really common in the JVM ecosystem to run private mirrors of Central.
I wasn’t aware that I was a commercial entity because I use github!
I think the point is that you are using a commercial entity to host your code. There is a bill for the code you have hosted, and you aren't the one paying for it.
If that was the point why did they write “commercially run”? That is explicitly about management, not hosting or single points of failure.
It's never a problem until it is all at once and you realize they hold all the keys.
Re:NPM Is Joining GitHub
by steveklabnik on Monday March 16, 2020 @08:08PM
(#22599816)
crates.io is not a commercial enterprise.
Confused why you think a service servings millions or billions of requests a day wouldn't require money to run. Do you think some grant magically appear out of thin air to pay for the servers, storage, bandwidth, and maintenance?
We've been doing fine here in reality, where most Linux distros, Perl's CPAN, Python's PyPi, RubyGems, etc. are all run by volunteers and donations. There is nothing special about NPM that requires it to be owned by a for-profit corporation. If Wikipedia and Archive.org can get by, I'm sure NPM can too.
Big leap from "servers cost money" to "a package manager requires a commercial entity". How are other language ecosystems and package managers operating, many without private companies attached, when they too are serving millions of requests a day?
Well the big ones are hosted by commercial enterprises, and the smaller ones don't have the scale of JavaScript so they don't require that level of support.
Which package manager has the scale of JS and isn't hosted by a commercial entity?
apt, pacman, pip, and others. You can have large scale systems that aren't dependent on a commercial entity.
I mean, the scale of pacman is miniscule compared to NPM. PyPI is hosted by AWS and Google.
Not sure how APT works in the linux community can really work for anyone else, but definitely worth a shot for sure.
Almost any other package repository is funded by donations from companies using them or a grant from an infrastructure provider.
PyPy runs on donated infrastructure that costs over 800K/month in hosting costs[1].
Not many non-commercial entities can afford that.
PyPI, not PyPy. For those confused over how PyPy could cost that much.
Compiling Python code isn't easy!
Yes you are correct of course - my mistyping!
Looks like they’re paying for bandwidth, aka “cloud pricing.” They could probably reduce those numbers significantly, but if it’s funded by a grant why bother?
Good reminder to use pip cache.
Why does NPM need to be funded as a commercial entity at all? What other open source library has a private company running its package manager? This one still boggles my mind.