News for hackers. Stuff that matters.
Re:Nix × IPFS – Milestone 1 by imhoguy on Tuesday September 29, 2020 @08:17AM (#24626759)
Open source plus distributed p2p file sharing is the killer combo. I don't get why public stuff like NPM, DEB or Docker registries haven't switched to use it as primary way of distribution.

P2P, such as IPFS and recently BitTorrent 2.0 with its hash tree per file, is the only free (as beer and speech) reliable way to host things online forever - at least as long there is the last of the veterans who keeps a copy and seeds it.

Re:Nix × IPFS – Milestone 1 by solarkraft on Tuesday September 29, 2020 @09:42AM (#24627682)
> I don't get why public stuff like NPM, DEB or Docker registries haven't switched to use it as primary way of distribution

2 of them are run by commercial entities. I don't think those are all that interested in reducing dependency on them.

Re:Nix × IPFS – Milestone 1 by whyrusleeping on Tuesday September 29, 2020 @11:03AM (#24628672)
Re:Nix × IPFS – Milestone 1 by akerro on Tuesday September 29, 2020 @01:55PM (#24630901)
>2 of them are run by commercial entities. I don't think those are all that interested in reducing dependency on them.

Docker is also removing some old images and putting limits on existing images, it's too expensive for them to host.

Re:Nix × IPFS – Milestone 1 by m463 on Thursday October 01, 2020 @06:50AM (#24649464)
or it could be a strategy to move more people to a pay model.
Re:Nix × IPFS – Milestone 1 by na85 on Tuesday September 29, 2020 @10:10AM (#24628004)
In a lot of the world, upload speeds are very limited and ISPs charge an arm and a leg if you exceed your upstream data caps.

On top of that, Bit torrent has a tendency to overwhelm routers and make them slow to a crawl.

Those are a lot of downsides to ask your users to accept just so you can cheap out on download servers.

Re:Nix × IPFS – Milestone 1 by takeda on Tuesday September 29, 2020 @01:27PM (#24630566)
Perhaps there are routers that don't handle well large amounts of data, but from my experience what typically is happening are two things:

1. the fairness in TCP is done per TCP connection, bittorrent opens many connections so they take over the whole bandwidth. You can limit number of connections or throttle the bandwidth used by BT (of course you will get slower speeds), maybe there are some other ways.

2. when you maximize your throughput ACKs might get dropped which slows everything down (I suspect that might be the issue you're talking about) again you could throttle or enable traffic shaping and give highest priority to ACKs. This worked well for me.

Re:Nix × IPFS – Milestone 1 by linsomniac on Tuesday September 29, 2020 @07:30PM (#24634283)
3. Bufferbloat. It's still a thing. dslreports has a bufferbloat score, one of my coworkers has been really impacted by it. http://www.dslreports.com/speedtest
Re:Nix × IPFS – Milestone 1 by takeda on Tuesday September 29, 2020 @08:03PM (#24634497)
I heard about bufferbloat, but I didn't think it was related to bittorrent maximizing throughput
Re:Nix × IPFS – Milestone 1 by linsomniac on Tuesday September 29, 2020 @11:06PM (#24635313)
It depends: packet loss due to bitorrent is a problem, but low loss and high latency makes it dramatically worse, and that is the bufferbloat cast. If you have 1-30 second pings (or I've seen even worse), when doing torrents or other heavy outbound traffic, it falls apart.
Re:Nix × IPFS – Milestone 1 by Dagger2 on Thursday October 01, 2020 @09:26AM (#24650545)
Even 1 second is already pretty severe bufferbloat, let alone 30 seconds. Your latency should increase by under 25-125ms even with upstream and downstream both running flat out.
Re:Nix × IPFS – Milestone 1 by imhoguy on Wednesday September 30, 2020 @01:02PM (#24640946)
But all this can be adjusted with concepts like pining, rate limiting, queuing, connection limits, even geo-fencing. I would gladly share % of my server's unlimited transfer to give back a tiny range of Debian or DockerHub packages which I have dowloaded and use. And I don't want to mirror the entire distros, which is condition to become an official host of distro mirror.
Re:Nix × IPFS – Milestone 1 by akerro on Tuesday September 29, 2020 @01:56PM (#24630917)
> In a lot of the world, upload speeds are very limited and ISPs charge an arm and a leg if you exceed your upstream data caps.

Not all 1st world countries have data caps. I pay ~20$ for month for 37mbps (15 upload) with no data caps (it's literally the cheapest plan I could find, if there was cheaper available, I would take it).

Re:Nix × IPFS – Milestone 1 by seniorivn on Wednesday September 30, 2020 @08:37AM (#24638123)
in Russia i have a ~$15 plan 1gbps (~700 realistically)up/down unfortunately it's clear that ISPs are not going to be able to upgrade their hardware nor keep prices low due to government interventions, government controlled ISPs competetion, mandated DPI censorship(paid by ISPs) so very soon internet in Russia will be as shitty as in most western countries that lack competition

P.S. until 2009-2014 russian internet was completely out of government sight. an average internet user in big cities had a choice of 3-10 ISPs competing prices etc

Re:Nix × IPFS – Milestone 1 by Avamander on Tuesday September 29, 2020 @11:30AM (#24629019)
With LPD, we could have two PCs upload to each other, it would be even beneficial to ISPs if everyone didn't have to download updates at a similar time from the furthest-away server. Especially in developing countries.
Re:Nix × IPFS – Milestone 1 by Avamander on Tuesday September 29, 2020 @08:23AM (#24626828)
I don't get why machines running the same Linux distribution couldn't actually do the same. BT2 would indeed work very well for this.
Re:Nix × IPFS – Milestone 1 by e12e on Tuesday September 29, 2020 @12:57PM (#24630183)
Nixos/guix might be a good fit for rootfs on ipfs? Comes with a local cache, and you could overlay local changes (eg: /etc/passwd)?
Re:Nix × IPFS – Milestone 1 by takeda on Tuesday September 29, 2020 @01:41PM (#24630723)
Actually not exactly rootfs on ipfs, but with NixOS is fairly easy to make it wipe data on reboot, here's an example article[1]. The idea is to only keep state we care about and rebuild the system on every boot.

[1] https://grahamc.com/blog/erase-your-darlings

Re:Nix × IPFS – Milestone 1 by shp0ngle on Tuesday September 29, 2020 @02:24PM (#24631231)
BitTorrent 2.0 is not recent, no.

The current bittorrent company is interested in chasing ICOs, I don’t think they work on new versions of the protocol anymore?

edit: yeah it’s from 2008

Re:Nix × IPFS – Milestone 1 by leppr on Tuesday September 29, 2020 @05:25PM (#24633188)
Going on a tangent but this article that came out today is a good read for anyone that didn't follow what happened to BitTorrent the company: https://www.theverge.com/21459906/bittorrent-tron-acquisitio...
Re:Nix × IPFS – Milestone 1 by capableweb on Tuesday September 29, 2020 @02:54PM (#24631540)
People are working on BitTorrent yes! libtorrents blog made a blog post quite recently (September 7th, 2020) - https://blog.libtorrent.org/2020/09/bittorrent-v2/
Re:Nix × IPFS – Milestone 1 by shp0ngle on Wednesday September 30, 2020 @08:01PM (#24645619)
That is libtorrent, unrelated to bittorrent, no? (organizationally)
Re:Nix × IPFS – Milestone 1 by capableweb on Thursday October 01, 2020 @07:26AM (#24649668)
Yeah, maybe you were explicitly talking about the company BitTorrent Inc.

"work on new versions of the protocol anymore?" made me think of the BitTorrent protocol in general, sorry for the misunderstanding.

I think you're right that the BitTorrent company aren't directly involved in the protocol work, they seem to be focused on trying to push their plagiarized cryptocurrency project.

Re:Nix × IPFS – Milestone 1 by cpuguy83 on Tuesday September 29, 2020 @11:00AM (#24628633)
Because changing things out is hard and the benefit of pulling in that much more code and complexity is dubious.

It seems like you could implement a p2p solution as a pull through registry... as in host a registry (even per node if you like) that Docker will use as a pull through cache, still speak the same distribution protocol, but have the registry use p2p to get the content.