Re:Suddenly, Hacker News is not the first result for 'Hacker...
by moe on Friday November 25, 2011 @08:54PM
(#3278841)
+1 for varnish. It's stupidly[1] fast and there shouldn't be much trickery required to deflect most of HN's traffic (e.g. ~10 sec expiry for "live" pages, infinite expiry for archived pages).
[1] 15k reqs/sec on a moderate box
If the cache has a copy of an article that is a few hours old it will just give that version to Googlebot while if it thinks a human is requesting the page then it will go to the backend and fetch the latest version.
https://www.varnish-cache.org/lists/pipermail/varnish-misc/2...