News for hackers. Stuff that matters.
Re:MinIO passes 1B cumulative Docker Pulls by teeray on Wednesday September 21, 2022 @10:17PM (#32934125)
It would be lovely to see Bittorrent in this space. Distributing an identical blob to N machines is squarely within its wheelhouse. All those CI machines would make wonderful seeders too.
Re:MinIO passes 1B cumulative Docker Pulls by danmcs on Thursday September 22, 2022 @12:18AM (#32934743)
Uber Engineering open-sourced Kraken [1], their peer-to-peer docker registry. I remember it originally using the BitTorrent protocol but in their readme they now say it is "based on BitTorrent" due to different tradeoffs they needed to make.

As far as I know there aren't any projects doing peer-to-peer distribution of container images to servers, probably because it's useful to be able to use a stock docker daemon on your server. The Kraken page references Dragonfly [2] but I haven't grokked it yet, it might be that.

It has seemed strange to me that the docker daemon I run on a host is not also a registry, if I want to enable that feature.

It's also possible that in practice you'd want your CI nodes optimized for compute because they're doing a lot of work, your registry hosts for bandwidth, and your servers again for compute, and having one daemon to rule them all seems elegant but is actually overgeneralized, and specialization is better.

1 https://github.com/uber/kraken

2 https://d7y.io/

Re:MinIO passes 1B cumulative Docker Pulls by scary-size on Thursday September 22, 2022 @02:27AM (#32935391)
The gaming industry has been using peer to peer downloads for a long time AFAIK. Unsure if they still do, pretty sure the Battle.net client did it.
Re:MinIO passes 1B cumulative Docker Pulls by smileybarry on Thursday September 22, 2022 @05:49AM (#32936720)
It was used in the WoW downloader pre-Battle.NET, they've moved off that and switched to CDNs (mostly Akamai IIRC) & dynamic patching, i.e.: Battle.NET computes differences and gets a diff.

Steam never used P2P downloading but they've had deals with ISPs and communities really early and made their own "CDN" full of caching servers everywhere, so if you waited a few hours to update, it might make it to your local server in time. Now I think they're on Akamai as well.

Re:MinIO passes 1B cumulative Docker Pulls by scary-size on Thursday September 22, 2022 @06:00AM (#32936800)
Oh, right, there was the WoW launcher before battle.net. I think I have another game in mind which used this, but I can’t really remember. Would love to get more insights on the economics of this (p2p vs. cdn).
Re:MinIO passes 1B cumulative Docker Pulls by smileybarry on Friday September 23, 2022 @01:48PM (#32955039)
WoW popularized it and a lot of “WoW killer” MMOs had P2P, but I never saw it “work” to be honest.