News for hackers. Stuff that matters.
Re:Snatch: A simple and fast download accelerator, in Rust by ycmbntrthrwaway on Saturday January 21, 2017 @10:45AM (#13450524)
Downloading from several mirrors at once makes sense, but using "download accelerators" to cheat on TCP congestion control is just wrong. Some mirrors will even ban you for making more than 4 connections at once.
Re:Snatch: A simple and fast download accelerator, in Rust by k0pernicus on Saturday January 21, 2017 @11:14AM (#13450665)
Hi, actually, this was an idea that we shared a few days ago, and I am agree with this. As I mentioned earlier in a next comment, Snatch is a side project that has been created for a presentation of Rust. Thank you for this comment - we will work on this idea soon :-)
Re:Snatch: A simple and fast download accelerator, in Rust by zkms on Sunday January 22, 2017 @12:07AM (#13453689)
Have you considered adding support for download-accelerating strategies like https://news.ycombinator.com/item?id=11842517?
Re:Snatch: A simple and fast download accelerator, in Rust by the8472 on Saturday January 21, 2017 @12:00PM (#13450890)
Indeed, but TCP's default congestion control might be suboptimal on some pipes.

Maybe an option to choose an alternative controller via socket options (linux and windows offer alternatives) would make sense in some situations.

Re:Snatch: A simple and fast download accelerator, in Rust by ycmbntrthrwaway on Saturday January 21, 2017 @04:07PM (#13452165)
> TCP's default congestion control might be suboptimal on some pipes

Default TCP congestion control is Reno, but Linux uses Cubic, increased initial CWND and all the other modern modifications. CDNs have their own proprietary congestion controls that are arguably better, but Google has already pushed its own congestion control (BBR) into Linux, so it is just a matter of time until it becomes new default. There is no reason to tamper with these settings unless you are operating Netflix/YouTube, some sort of CDN or unusual (satellite) link.

> Maybe an option to choose an alternative controller via socket options (linux and windows offer alternatives) would make sense in some situations.

Only the server congestion control matters. Client simply acknowledges whatever it receives as soon as possible and waits for more data.

Re:Snatch: A simple and fast download accelerator, in Rust by paulddraper on Saturday January 21, 2017 @11:43PM (#13453617)
> Only the server congestion control matters. Client simply acknowledges whatever it receives as soon as possible and waits for more data.

To be clear, that's true if the server is the only one transmitting data. (I.e. for a download)

Re:Snatch: A simple and fast download accelerator, in Rust by gtdawg on Saturday January 21, 2017 @04:50PM (#13452333)
Some users can have more than one external IPv4/IPv6 address (One for each Internet connection, if they have multiple). In order to use all internet connections for the download, they would have to initiate multiple tcp sockets.

What kind of logic should this app use to determine how many external addresses are actually available in the case of IPv4? As you say, it is more "tcp fair" to have only one TCP connection per client/server relationship in normal cases.

Just some random thoughts. Apologies if it's difficult to follow. I can reply to any questions if anything is confusing.

Re:Snatch: A simple and fast download accelerator, in Rust by ycmbntrthrwaway on Sunday January 22, 2017 @08:14AM (#13455042)
> In order to use all internet connections for the download, they would have to initiate multiple tcp sockets.

Opening one connection per interface is still a violation of TCP design. Look into MPTCP RFCs. It is specifically designed to use multiple connections as if you had only one. Developers of the standard made sure you don't have any advantage compared to someone using only one connection.

Two connections over two different interfaces will meet somewhere on the bottleneck near the mirror so you still have the advantage of multiple TCP connections.

Sure MPTCP is not ready yet, but multipath QUIC will be deployed by the end of this year so all your Chrome downloads from google drive will utilize multiple connections.

Re:Snatch: A simple and fast download accelerator, in Rust by milankragujevic on Sunday January 22, 2017 @01:59AM (#13454033)
Google does the banning, but they limit their download speed to some slow speed and I have a 1 Gbps server which I use to download stuff, so I often open up to 32 connections to their server just to download at a normal rate. They have the bandwidth, why do the limit the speed is beyond me. You can see this on the googlevideo.com domain, for Google Drive transcoded videos, thought not for YouTube.