Re:Downloading a file regularly - how hard can it be?
by rmc on Monday April 09, 2012 @12:55PM
(#3817800)
Server load goes really high on that day, and if you get more popular, you'll need more servers and hence more money.
Re:Downloading a file regularly - how hard can it be?
by tantalor on Friday April 13, 2012 @12:41PM
(#3837570)
Maybe it wasn't clear, but in this case the load is still distributed evenly across all days. I always download on Monday, you always download on Tuesday. The author suggests that this is not desirable.
FTA,
The “Total” row looks very nice, it’s exactly what we would like to have. However, our download interval matches the number of days now, with the effect that people always download on the same weekday (and those currently downloading on a Monday will continue downloading on a Monday).
Re:Downloading a file regularly - how hard can it be?
by kijin on Monday April 09, 2012 @02:45PM
(#3818542)
Use a CDN? It's not like the list is tailored to anybody.
Re:Downloading a file regularly - how hard can it be?
by rb2k_ on Monday April 09, 2012 @12:56PM
(#3817808)
Isn't that something that nginx/varnish should easily be able to handle? It is just a static file download after all...
Re:Downloading a file regularly - how hard can it be?
by ComputerGuru on Monday April 09, 2012 @01:52PM
(#3818247)
CPU and bandwidth are entirely different issues. Sure, nginx can handle the processing. But do you have the piping to match?
A run of the mill dedicated server has a 100mbit uplink. Do the math. (Hint: it's easy to saturate in no time).
Re:Downloading a file regularly - how hard can it be?
by prostoalex on Monday April 09, 2012 @01:57PM
(#3818277)
Has anybody tried https://developers.google.com/speed/pagespeed/service for this?
Re:Downloading a file regularly - how hard can it be?
by oconnor0 on Monday April 09, 2012 @03:04PM
(#3818632)
This is just downloading a single static text file so there's nothing to optimize.
Re:Downloading a file regularly - how hard can it be?
by onli on Monday April 09, 2012 @01:15PM
(#3817942)
Even serving a static file is a burden on the server when you have millions of requests.
Re:Downloading a file regularly - how hard can it be?
by rb2k_ on Monday April 09, 2012 @01:33PM
(#3818084)
What are the exact numbers?
A quote from: http://wiki.nginx.org/Main :
> I currently have Nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a single server. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6).
Re:Downloading a file regularly - how hard can it be?
by onli on Monday April 09, 2012 @02:07PM
(#3818337)
https://easylist.adblockplus.org/blog/2011/09/01/easylist-st... is the first thing i found, talking about 11.5 million total users and 80% of them using easylist, 9.2 million. According to the blogpost, the still existing (i just noticed that this was after the update-behaviour change) monday-peak was 118.5% of the expected (week total / 7) - 73 million download in august, so (73 / 4 / 7) * 1.185 = 3 million for that specific list and 3.75 for all of them. Hope i didn't miscalculate ^^
Add to that the growth till than. The easylist.txt seems to have a size of 528kb.
Re:Downloading a file regularly - how hard can it be?
by j_s on Monday April 09, 2012 @05:31PM
(#3819345)
+ SSL
Re:Downloading a file regularly - how hard can it be?
by zackzackzack on Monday April 09, 2012 @12:54PM
(#3817794)
Everybody downloading at once makes for slow servers. The author probably could pay much less in hosting costs if everybody downloaded the file along a uniform disturbance.
What's so bad about that?