> Why is there a need for so many varnish servers?
Probably because a lot of their assets are static image files.
Ah, so its a filesize issue? ie- they have xxx TB of storage and they obviously need one cache/serving process per machine.
I was just curious why there was so many cache process's needed but this makes sense.
These guys might make good use of the open source backblaze specs- I know we are looking into this for some storage of video/image data : http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v...
We use Varnish because it's configurable, and have 6-8x more app servers than cache servers. In the future we'll scale back the Varnish layer, since more and more of what it does will be handled via CDN.
I only ask because we run nginx as a cache and the vast majority of our content is served directly from it ( almost 80% of page loads are served via cache ) and a single nginx process handles everything for us.