News for hackers. Stuff that matters.
Re:Are Package Registries Holding Open-Source Hostage? by autarch on Wednesday November 11, 2020 @02:02PM (#25061621)
This article ignores the fact that for many languages, the package repository is maintained as free software by volunteers (sometimes with funding from a foundation). This includes Perl, Python, Ruby, Rust, and many others.

NPM is the odd one out, really. I don't think letting one company control a language ecosystem's single package registry is a great idea, for all the reasons that the author notes!

Re:Are Package Registries Holding Open-Source Hostage? by takluyver on Wednesday November 11, 2020 @02:21PM (#25061834)
I agree, though it's worth noting that while volunteers can maintain the software and administer the indexes, they also rely on infrastructure provided by big corporations. E.g. the Python Package Index runs on a CDN provided by fastly, which serves hundreds of TB per day. I very much doubt the non-profit Python Software Foundation could afford that bandwidth if it wasn't an in-kind donation.
Re:Are Package Registries Holding Open-Source Hostage? by luhn on Wednesday November 11, 2020 @03:46PM (#25062774)
It's a couple petabytes, Michael. What could it cost, $10?

Seriously though, Fastly's donation of their CDN service is generous and eases the burden on the PSF, but if push came to shove they could definitely afford the bandwidth. In 2018 they had a net income of half a million.

Re:Are Package Registries Holding Open-Source Hostage? by di on Thursday November 12, 2020 @11:49AM (#25071689)
Hi, PyPI maintainer and PSF director here.

There's absolutely no way the PSF could afford PyPI's bandwidth out of pocket. Last I checked our "bill" from Fastly would be close to $1.5M/month.

Also given that PyPI is critical infrastructure for millions of people and software projects, anything cheaper would not really cut it.

Re:Are Package Registries Holding Open-Source Hostage? by tedivm on Wednesday November 11, 2020 @08:18PM (#25065223)
As of March PyPi was pushing out 300TB a day through its CDN. Ignoring the "off the shelf price" of $0.12/gb and assuming they negotiate a bulk discount driving them down to $0.05/gb that's still $15,000 a day (or just shy of $5.5 million a year). Their net income in 2018 would cover less than 10% of that bill.
Re:Are Package Registries Holding Open-Source Hostage? by true_religion on Wednesday November 11, 2020 @08:25PM (#25065291)
Well.... they couldn't afford Fast.ly, or Akamai, or Brightcove, but they could afford any tier-2 or tier-3 CDN.

For example CDN77, would start you off at 0.016/gb if you have more than 100TB per month, without any negotiation.

If you have 300TB per day, you surely can negotiate sub-cent pricing somewhere.

No, the costs are manageable.

I think the charitable/community projects setup for developers, need to act like non-profits in other sectors and actively seek out donations.

Re:Are Package Registries Holding Open-Source Hostage? by takluyver on Thursday November 12, 2020 @04:57AM (#25068252)
I assume there's some difference in e.g. speed or reliability between Fastly (8 c/GB) and CDN77 (1.6 c/GB)? Even if you can negotiate it to 0.8 c/GB, you're still talking about a huge piece of the PSF's budget. They would have to either cut other expenditure (e.g. making grants) dramatically, or find a lot of new income (continuously, not just a one-off donation drive). And PyPI's bandwidth is growing rapidly [1].

If PyPI didn't have sponsor providing bandwidth, I'd guess it would implement some form of rate limiting and encourage people to mirror/cache packages much more to reduce load. I don't think it would die completely, but it would be less convenient and still cost the PSF a fair bit of money.

[1] https://twitter.com/di_codes/status/1235707819955032069

Re:Are Package Registries Holding Open-Source Hostage? by takluyver on Thursday November 12, 2020 @05:08AM (#25068313)
The PSF's tax returns are published on python.org, and 'revenue less expenses' for 2018 was just under $280k. IANA accountant, so maybe that's the wrong line to look at.
Re:Are Package Registries Holding Open-Source Hostage? by webology on Thursday November 12, 2020 @02:58PM (#25074154)
Hi, I'm a PSF Director and the PSF's Treasure as of this year. For transparency, our tax returns as of 2018 (and soon 2019) are up on https://www.python.org/psf/records/

You are right that this donation does not show up in our tax filings because they provide it to any OSS project.

Re:Are Package Registries Holding Open-Source Hostage? by autarch on Thursday November 12, 2020 @04:59PM (#25075674)
Also, it's not income that you would put in your 990s. Similarly, the donor cannot deduct the expense of the donation.
Re:Are Package Registries Holding Open-Source Hostage? by webology on Thursday November 12, 2020 @09:31PM (#25078155)
Correct on both points. (I do consider it a potential liability though.)
Re:Are Package Registries Holding Open-Source Hostage? by autarch on Wednesday November 11, 2020 @02:26PM (#25061898)
Yes, that's a good point. The Perl repo and services are also relying on donations from various companies. That said, it's easier to switch donors than it is to switch repositories.
Re:Are Package Registries Holding Open-Source Hostage? by ForHackernews on Wednesday November 11, 2020 @05:32PM (#25063804)
If necessary, it seems like it'd be easy for any of these package registries to ~~blackmail~~ encourage big companies into donating infrastructure.

"If you don't support us, we might accidentally forget to audit our packages and feed malware into your build pipelines. It'd truly be a shame..."