What about integrating (and encouraging users to opt into) peer-to-peer distribution? If I understand correctly, this would work for Guix and Nix.
I'm not using Guix, but if I was I would be happy to do this. I have no data cap on my home internet, and my PC is usually on.
There was early work on using IPFS as a substitute distribution mechanism, but the API changed and nobody has picked up the existing work yet. But I agree that this would be a great feature and I hope someone will feel motivated to pick up and assemble the pieces.
What are the security implications of this?
The implementation requires that you have at least one authorized substitute server advertising the same hash.
In simplified terms, if ci.guix.gnu.org advertises a substitute for /gnu/store/abc123-foo, with the checksum "xyz789" (and the cryptographic signature of that advertisement checks out), your daemon can safely download that file over P2P.
Ah, I think I'm misunderstanding the intent here. Clearly P2P distribution of checksummed binaries can be safe, I was just wondering if there were a solution to the build farm being behind. It seems like you can't really trust the first build of any artifact unless it comes from a central source.
There have been discussions of an "N of P" distribution, i.e. if 80% of available peers (or substitute servers) advertise the same build result, then treat it as safe.
I expect that both will be implemented, and the choice left up to the user.
Unless I'm massively underestimating the number of Guix users, that seems quite easy to exploit.
With regards to downloading speeds, the issue seems to be downloading many packaging as opposed to large packages. Each package incurs some noticeable overhead, which is painful when upgrades consist of dozens or hundreds of upgraded packages.
Slow download appears to depend a lot on location to an extent that we suspect peering problems. The main build farm's head node is connected to the internet from a well-connected research campus with dual 10G. Multiple tests have concluded that the configuration from server-side software, to network hardware of the server, down to networking in the data centre is sane.
Maybe we'll have to bite the bullet and (again) use a CDN to better serve users that inexplicably experience slow connections.