News for hackers. Stuff that matters.
Re:Downloading a file regularly - how hard can it be? by rb2k_ on Monday April 09, 2012 @12:56PM (#3817808)
Isn't that something that nginx/varnish should easily be able to handle? It is just a static file download after all...
Re:Downloading a file regularly - how hard can it be? by ComputerGuru on Monday April 09, 2012 @01:52PM (#3818247)
CPU and bandwidth are entirely different issues. Sure, nginx can handle the processing. But do you have the piping to match?

A run of the mill dedicated server has a 100mbit uplink. Do the math. (Hint: it's easy to saturate in no time).

Re:Downloading a file regularly - how hard can it be? by prostoalex on Monday April 09, 2012 @01:57PM (#3818277)
Re:Downloading a file regularly - how hard can it be? by oconnor0 on Monday April 09, 2012 @03:04PM (#3818632)
This is just downloading a single static text file so there's nothing to optimize.
Re:Downloading a file regularly - how hard can it be? by onli on Monday April 09, 2012 @01:15PM (#3817942)
Even serving a static file is a burden on the server when you have millions of requests.
Re:Downloading a file regularly - how hard can it be? by rb2k_ on Monday April 09, 2012 @01:33PM (#3818084)
What are the exact numbers?

A quote from: http://wiki.nginx.org/Main :

> I currently have Nginx doing reverse proxy of over tens of millions of HTTP requests per day (thats a few hundred per second) on a single server. At peak load it uses about 15MB RAM and 10% CPU on my particular configuration (FreeBSD 6).

Re:Downloading a file regularly - how hard can it be? by onli on Monday April 09, 2012 @02:07PM (#3818337)
https://easylist.adblockplus.org/blog/2011/09/01/easylist-st... is the first thing i found, talking about 11.5 million total users and 80% of them using easylist, 9.2 million. According to the blogpost, the still existing (i just noticed that this was after the update-behaviour change) monday-peak was 118.5% of the expected (week total / 7) - 73 million download in august, so (73 / 4 / 7) * 1.185 = 3 million for that specific list and 3.75 for all of them. Hope i didn't miscalculate ^^ Add to that the growth till than. The easylist.txt seems to have a size of 528kb.
Re:Downloading a file regularly - how hard can it be? by j_s on Monday April 09, 2012 @05:31PM (#3819345)
+ SSL