News for hackers. Stuff that matters.
Re:Why HN was slow and how Rtm fixed it by rarrrrrr on Wednesday January 19, 2011 @03:49PM (#2121225)
Since no one has mentioned it yet - Varnish-cache.org, written by a FreeBSD kernel hacker, has a very nice feature, in that it will put all overlapping concurrent requests for the same cacheable resource "on hold", only fetch that resource once from the backend, then serve the same copy to all. Nearly all the expensive content on HN would be cacheable by varnish. Then you can get it down to pretty close to "1 backend request per content change" and stop worrying about how arbitrarily slow the actual backend server is, how many threads, how you deal with the socket, garbage collection, and all that.
Re:Why HN was slow and how Rtm fixed it by j_baker on Wednesday January 19, 2011 @04:05PM (#2121274)
Can't you only use varnish for mostly non-dynamic content? Like for example, wouldn't the fact that it displays my username and karma score at the top of the page make it so that you couldn't use varnish (or at least make it more difficult)?
Re:Why HN was slow and how Rtm fixed it by seiji on Wednesday January 19, 2011 @04:23PM (#2121328)
Check out http://www.varnish-cache.org/trac/wiki/ESIfeatures

There could be a private internal URL to just return username and karma to populate the user info header.

Re:Why HN was slow and how Rtm fixed it by j_baker on Wednesday January 19, 2011 @06:23PM (#2121733)
Doesn't that kind of defeat the purpose though? The point of using varnish is that it keeps you from having to access the backend altogether. This is getting into an area where something like memcache might be more appropriate.
Re:Why HN was slow and how Rtm fixed it by danudey on Wednesday January 19, 2011 @06:34PM (#2121765)
Well, the point of using varnish is to keep you from having to access the backend any more than is absolutely necessary. It's incredibly trivial to generate HTML showing a user's username and karma, and even if it weren't it could be stored in memcached. Generating the front page, the comments pages, etc. is the hard part, and varnish can keep that from being generated any more than is necessary.
Re:Why HN was slow and how Rtm fixed it by j_baker on Wednesday January 19, 2011 @07:17PM (#2121924)
Of course, but I seem to recall pg writing at some point that one of the goals of HN being to prove that "slow" languages can scale using caching. I assume, therefore, that he already has caching of some kind in place for those things. If varnish isn't going to save an access to the server (which seems to be the primary thing that's slowing things down), what value is varnish providing above what pg already has in place?
Re:Why HN was slow and how Rtm fixed it by danudey on Wednesday January 19, 2011 @07:21PM (#2121932)
The requests won't queue up as badly because the server will be able to clean out 'simple' requests in a much lower time than generating much larger pages. They won't queue up as much because the requests take less time to handle, so they can be cleared out faster than they come in (compared to larger requests that queue up faster than they can be handled).
Re:Why HN was slow and how Rtm fixed it by dminor on Wednesday January 19, 2011 @04:23PM (#2121326)
Yes, you'd have to somehow separate the dynamic content from the static content so they could be fetched in different requests and then combined (probably via ajax). If it's just your username and karma then it's simple enough, but if the comments are displayed differently for different people then it could be bear.
Re:Why HN was slow and how Rtm fixed it by rarrrrrr on Wednesday January 19, 2011 @05:44PM (#2121585)
I think the only way PG would try this is if it's clear that javascript, client side hacks, etc are not required -- and I think that's the case.

Aside from the header, there's only a relatively small number of variations for any given content, right? Showdead, ability to downvote, etc? So, each of these variations gets a distinct ESI URL. Like /item_$showdead_$downvote_$etc right in the internal URL, so any combination of these is a distinct URL. Only the first user to hit any particular combination would result in a request to the backend, and that could remain in cache until its content changed. No wizardry required.

Re:Why HN was slow and how Rtm fixed it by yummyfajitas on Wednesday January 19, 2011 @05:13PM (#2121485)
Comments are displayed differently for different people.

I once posted a comment which was immediately invisible to everyone besides me - I'm guessing it was marked as spam for some reason, but left visible to me so I think I successfully posted it.

Re:Why HN was slow and how Rtm fixed it by nuclear_eclipse on Wednesday January 19, 2011 @04:21PM (#2121319)
Reverse proxies won't work for HN, because requests for the same resource from multiple users can't use the same results. Not only are certain bits of info customized for the user (like your name/link at the top), but even things like the comments and links are custom per user.

Things like users' showdead value, as well as whether the user is deaded, can drastically change the output of each page. Eg, comments by a deaded user won't show as dead to that user, but they will for everyone else...

Re:Why HN was slow and how Rtm fixed it by jjoe on Wednesday January 19, 2011 @05:34PM (#2121544)
There's cookie-based caching in Varnish (and in some other proxy caches too). Essentially, the key is going to be made of the usual hash + the cookie like this:

sub vcl_hash { set req.hash += req.http.cookie; }

What this means is that the cache is per-logged-in-user and pretty much personalized. The server's going to need a lot more RAM than usual. You can set a low TTL on the cache entries so they're flushed and not kept in memory indefinitely. But the performance boost is great.

This is not recommended as an always-on measure. We wrote an entry about accomplishing something similar w/ python&varnish. Here it is if you're interesting in reading about it: http://blog.unixy.net/2010/11/3-state-throttle-web-server/

Regards

Re:Why HN was slow and how Rtm fixed it by piotrSikora on Thursday January 20, 2011 @12:09AM (#2122733)
Of course it will work. The whole point of reverse-proxy is to buffer slow requests and send them fast over LAN to your back-end servers that cannot handle high concurrency efficiently.

The FreeBSD's accept_filter() used by Rtm does more or less that (you can think of it as of reverse-proxy in the kernel), but it only works for plain HTTP and HEAD/GET methods.

Re:Why HN was slow and how Rtm fixed it by aonic on Wednesday January 19, 2011 @04:57PM (#2121430)
Varnish supports edge side includes. The header bar could be an ESI and the rest of the page could be cached
Re:Why HN was slow and how Rtm fixed it by nuclear_eclipse on Wednesday January 19, 2011 @05:37PM (#2121555)
> and the rest of the page could be cached

Except they can't, for the reasons I mentioned above. Eg, if my account is deaded, when I view a thread with one of my own comments, it looks different than if someone else was viewing that some thread, especially for those of us with or without showdead checked in our profiles.

Its not as straightforward as you would like it to be.

Re:Why HN was slow and how Rtm fixed it by aonic on Wednesday January 19, 2011 @07:14PM (#2121915)
Special cookies could be set for dead users and users who enable showdead to bypass the cache.

For example, one of the sites I run has about 50K pageviews/day by logged in users, and another 600K pageviews/day by anonymous users coming from referrals or search engines. Logged in users have similar customization options so we bypass cache for these users by detecting a cookie.

Obviously going the cache route would require some changes to how things are setup, its not a turn-key solution. But the insignificant amount of changes are well worth it for most content sites, but for a user generated content site like HN it would also depend on how the TTLs and cache purging are setup.

Re:Why HN was slow and how Rtm fixed it by nitrogen on Wednesday January 19, 2011 @07:47PM (#2122019)
The majority of requests probably come from live accounts in good standing or from people not even logged in, so the majority of requests could still be cached.
Re:Why HN was slow and how Rtm fixed it by nkurz on Thursday January 20, 2011 @01:05AM (#2122860)
Interesting point: what percentage of viewers are logged in? I was presuming it was high, but I guess I really don't know.
Re:Why HN was slow and how Rtm fixed it by rfugger on Wednesday January 19, 2011 @04:26PM (#2121335)
To make this work you could do all the per-user stuff with javascript and ajax calls in the browser. It would be quite a bit of revamping though.
Re:Why HN was slow and how Rtm fixed it by Aaronontheweb on Wednesday January 19, 2011 @04:53PM (#2121419)
So, you can't do donut caching in Varnish?
Re:Why HN was slow and how Rtm fixed it by esbcupper on Thursday January 20, 2011 @02:03AM (#2122946)
Instead of having concurrent requests wait for the single request to the backend it's almost always better to use stale-while-revalidate (http://tools.ietf.org/html/draft-nottingham-http-stale-while...). AFAIK both varnish and squid support this.
Re:Why HN was slow and how Rtm fixed it by dauphin on Wednesday January 19, 2011 @04:00PM (#2121261)
You clearly don't understand the problem. Even mod_pagespeed or memcached would be more appropriate here: They are rate-limited by the LISP kernel anyway (we are talking about dynamic content here).
Re:Why HN was slow and how Rtm fixed it by rarrrrrr on Wednesday January 19, 2011 @05:46PM (#2121593)
Varnish sits in front of the backend, responding to all requests that it has cached content for directly, without bothering the backend at all. It lives higher in the stack than than pagespeed or memcache, and not limited by the backend's speed in any way.