Re:Downloading a file regularly - how hard can it be?
by rb2k_ on Monday April 09, 2012 @12:56PM
(#3817808)
Isn't that something that nginx/varnish should easily be able to handle? It is just a static file download after all...
[ Parent
| on HN ]
Re:Downloading a file regularly - how hard can it be?
by ComputerGuru on Monday April 09, 2012 @01:52PM
(#3818247)
CPU and bandwidth are entirely different issues. Sure, nginx can handle the processing. But do you have the piping to match?
A run of the mill dedicated server has a 100mbit uplink. Do the math. (Hint: it's easy to saturate in no time).
Re:Downloading a file regularly - how hard can it be?
by prostoalex on Monday April 09, 2012 @01:57PM
(#3818277)
Has anybody tried https://developers.google.com/speed/pagespeed/service for this?
Re:Downloading a file regularly - how hard can it be?
by oconnor0 on Monday April 09, 2012 @03:04PM
(#3818632)
This is just downloading a single static text file so there's nothing to optimize.
Re:Downloading a file regularly - how hard can it be?
by onli on Monday April 09, 2012 @01:15PM
(#3817942)
Even serving a static file is a burden on the server when you have millions of requests.
Re:Downloading a file regularly - how hard can it be?
by rb2k_ on Monday April 09, 2012 @01:33PM
(#3818084)
What are the exact numbers?
A quote from: http://wiki.nginx.org/Main :
> I currently have Nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a single server. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6).
Re:Downloading a file regularly - how hard can it be?
by onli on Monday April 09, 2012 @02:07PM
(#3818337)
https://easylist.adblockplus.org/blog/2011/09/01/easylist-st... is the first thing i found, talking about 11.5 million total users and 80% of them using easylist, 9.2 million. According to the blogpost, the still existing (i just noticed that this was after the update-behaviour change) monday-peak was 118.5% of the expected (week total / 7) - 73 million download in august, so (73 / 4 / 7) * 1.185 = 3 million for that specific list and 3.75 for all of them. Hope i didn't miscalculate ^^
Add to that the growth till than. The easylist.txt seems to have a size of 528kb.