News for hackers. Stuff that matters.

Ask HN: Has HN been slow/spotty loading for you?

I love HN. I love it that PG built it himself. It's not easy. I applaud him for it. This isn't a complaint but an inquiry.

The last couple days (especially around peak times) HN seems to have slowed dramatically. Pages take multiple seconds (think, 10 to 20) to get served and even the up/down vote images fail to load.

Is it just me?

( 42 comments | 81 points | on HN )
Re:Ask HN: Has HN been slow/spotty loading for you? by smackfu on Wednesday November 17, 2010 @12:38PM (#1914945)
I've had the images fail to load, as well as the stylesheet (so all the text comes up as Times New Roman.)
Re:Ask HN: Has HN been slow/spotty loading for you? by kindly on Wednesday November 17, 2010 @02:31PM (#1915396)
I think it may be due to failing dns lookup. It has been for me anyway. The images and css are on ycombinator.com and not news.ycombinator.com.
Re:Ask HN: Has HN been slow/spotty loading for you? by andreas_bak on Wednesday November 17, 2010 @01:19PM (#1915106)
It is slow for me as well. It seems that page needs some optimizations in order to be served and rendered faster when the server is under heavy loads.

1. I checked responses using FireBug plugin and it seems that pages are served uncompressed. For example the front page of the site (only html) is 31K uncompressed, with gzip compression it becomes 6.3K (80% smaller size means 80% less bandwidth to use). Most of web servers support compression.

2. All pages seem to be generated on every request, e.g. no HTTP cache is used on server side. Because I did not found any response headers set by caches. If for example, each page was cached just for 1 to 5 seconds it will reduce the stress on database and cpu significantly. On the other hand it will not impact user experience, because cached version is short lived and fresh results will appear on time. For example the front page is the most requested page of the site, assume that it is requested 100 times per second which means 100 queries to the database etc. If this page is cached for 1 second queries and html composition is reduced to 1 time per second. Personally, HTTP caching on server side is my favorite choice because you don't need to modify the program to use it. (super-fast Varnish cache is the most flexible solution I found).

Re:Ask HN: Has HN been slow/spotty loading for you? by smackfu on Wednesday November 17, 2010 @01:43PM (#1915210)
Wouldn't server side caching run into issues since the page is different for each user? For instance, all the vote up/down buttons seem to have your username hardcoded into them as the href.
Re:Ask HN: Has HN been slow/spotty loading for you? by andreas_bak on Wednesday November 17, 2010 @02:13PM (#1915327)
Yes you are right. I did not notice it. HTTP cache is effective on resources that are same for all users.

One possible solution is to decouple the personalized data and load it separately using AJAX. Votes are already implemented in a similar way.

Generally, I am proponent of HTTP caching. It requires some modifications of web sites in order to be effective but still the whole system remains less complicated, compared to implementations of cache on back-end (like `memcached').

For example reddit, which is similar to HN, is in constant struggle with their cache subsystems (http://blog.reddit.com/2010/05/reddits-may-2010-state-of-ser...). I believe it can be avoided by keeping cache and web site code apart.

Re:Ask HN: Has HN been slow/spotty loading for you? by smackfu on Wednesday November 17, 2010 @02:17PM (#1915344)
I know rails has the concept of caching HTML snippets, so you still have the page construction cost, but you avoid the database calls.
Re:Ask HN: Has HN been slow/spotty loading for you? by preek on Wednesday November 17, 2010 @02:05PM (#1915300)
Yes, it would.

I don't know how many people are not logged in, though. Those pages could be cached for a minute or so.

Re:Ask HN: Has HN been slow/spotty loading for you? by _b8r0 on Wednesday November 17, 2010 @11:41AM (#1914756)
I can't find the HN story, but I seem to remember that PG was rebooting the server quite regularly. It must be pretty difficult when you're looking at having to toss up continuing to use the language with it's limitations or start looking at changes to Arc.

Maybe a front-end cache like nginx would help?

Re:Ask HN: Has HN been slow/spotty loading for you? by wwortiz on Wednesday November 17, 2010 @02:30PM (#1915392)
As stated elsewhere it really seems to be a problem with the static content hosted on ycombinator.com (maybe they aren't being cached very well browser side).

I don't know how well something like a front end cache would do to speed up content delivery as I suspect news.yc has a lot more users logged in than normal websites which might take away the advantages of a front end cache.

Re:Ask HN: Has HN been slow/spotty loading for you? by sudont on Wednesday November 17, 2010 @04:44PM (#1915873)
Webkit's resource tracker shows that most of the static content is taking anywhere from 10-14 seconds to be requested, and then loading with a 304 not modified.

The page, from the news.ycombinator subdomain is loading quick, as is the tracking code. (180ms latency)

Firefox works the best for me, as it seems to just time out and not request anything.

Re:Ask HN: Has HN been slow/spotty loading for you? by singer on Wednesday November 17, 2010 @12:57PM (#1915016)
10 to 20 seconds? That's pretty speedy. I've seen pages take minutes to load. I've noticed the slowness you speak of for a couple of weeks now.
Re:Ask HN: Has HN been slow/spotty loading for you? by xiongchiamiov on Wednesday November 17, 2010 @04:44PM (#1915872)
Depends on your location and your pipe, I suppose. I expect simple pages like this to be returned and rendered within a second.
Re:Ask HN: Has HN been slow/spotty loading for you? by lukeqsee on Wednesday November 17, 2010 @01:21PM (#1915115)
That is slow for the HTML document to be returned (not rendered, but just returned).
Re:Ask HN: Has HN been slow/spotty loading for you? by pig on Wednesday November 17, 2010 @04:02PM (#1915733)
He is saying that there are HN pages that take minutes to load.
Re:Ask HN: Has HN been slow/spotty loading for you? by alt_ on Wednesday November 17, 2010 @01:02PM (#1915041)
news.ycombinator.com is loading in sub-seconds, but it uses stylesheets and images hosted on ycombinator.com (a separate server) - which is taking 20s+ to respond.
Re:Ask HN: Has HN been slow/spotty loading for you? by pbogdan on Wednesday November 17, 2010 @04:58PM (#1915921)
It's been slow for me as well past 1-2 weeks and yes, it's problem with static files - they take 20-30s to load.

Caching seems to be fine - I'm getting 304s for stylesheets and images (and they are served from local cache according to firebug and chrome dev. tools) which means it actually takes 20-30s of wait time for the server to just return the headers.

I wonder if it might be problem with Keep-Alive which is on (with 15 sec. timeout I think) - there's probably bunch of apache processes / threads sitting idle waiting for connections to close and with the amount of traffic hn gets I would imagine there's quite a few of them.

Re:Ask HN: Has HN been slow/spotty loading for you? by smackfu on Wednesday November 17, 2010 @01:46PM (#1915228)
Makes me wonder if something is off with the caching directives on the static content.
Re:Ask HN: Has HN been slow/spotty loading for you? by Scott_MacGregor on Wednesday November 17, 2010 @11:36AM (#1914741)
Yes, I noticed it too.

I don't know what the backend is, but if it's on a single shared machine it might be too many inquires on the hardware at once. Maybe ask the hosting company to move it to a machine with a lower load.

Re:Ask HN: Has HN been slow/spotty loading for you? by mrduncan on Wednesday November 17, 2010 @12:22PM (#1914898)
As of 2 years ago, it was hosted on a "3.0 GHz Core whatever, 12 GB RAM, 64-bit FreeBSD 7.1" - it's certainly not a shared machine. (http://news.ycombinator.com/item?id=516122)

rtm is the sysadmin for HN (yes, that rtm: http://en.wikipedia.org/wiki/Robert_Tappan_Morris).

Re:Ask HN: Has HN been slow/spotty loading for you? by BrandonM on Wednesday November 17, 2010 @12:47PM (#1914976)
> rtm is the sysadmin for HN (yes, that rtm: http://en.wikipedia.org/wiki/Robert_Tappan_Morris)

It always makes me laugh when someone with no knowledge of a website offers advice to that site's sysadmin. It's especially funny when that sysadmin is of the calibre of RTM.

Re:Ask HN: Has HN been slow/spotty loading for you? by pyre on Wednesday November 17, 2010 @01:34PM (#1915175)
IIRC, pg said that ycombinator.com (which is where all of HN's static content is served from) was on shared hosting.
Re:Ask HN: Has HN been slow/spotty loading for you? by mrduncan on Wednesday November 17, 2010 @01:40PM (#1915201)
It looks like ycombinator.com is hosted by SliceHost (http://www.whoishostingthis.com/ycombinator.com).
Re:Ask HN: Has HN been slow/spotty loading for you? by pbogdan on Wednesday November 17, 2010 @04:47PM (#1915885)
It seems to be on SliceHost (if nothing changed in the mean time) - http://news.ycombinator.com/item?id=1283430.
Re:Ask HN: Has HN been slow/spotty loading for you? by lukeqsee on Wednesday November 17, 2010 @11:40AM (#1914753)
It's an arc backend. (http://arclanguage.org/)
Re:Ask HN: Has HN been slow/spotty loading for you? by iamelgringo on Wednesday November 17, 2010 @02:21PM (#1915364)
It is running slow. YC is doing interviews this week, so PG/RTM are probably chatting to noob founders instead of looking at HN server stats.
Re:Ask HN: Has HN been slow/spotty loading for you? by scdlbx on Wednesday November 17, 2010 @12:09PM (#1914852)
I have not had any noticeable problems in the past week.
Re:Ask HN: Has HN been slow/spotty loading for you? by kreek on Wednesday November 17, 2010 @12:24PM (#1914906)
Pages load fine for me, however submitting a comment is taking longer than usual.
Re:Ask HN: Has HN been slow/spotty loading for you? by cosmicray on Wednesday November 17, 2010 @01:18PM (#1915098)
I'm looking at this page in Safari inspector. The only element that took more than milliseconds to load is prowidget.php (but that may be related to its size, and my slow ass connection).

inspector has flagged 3 elements with "You could save bandwidth by having your web server compress this transfer with gzip or zlib": item, news.css and propres.php

Re:Ask HN: Has HN been slow/spotty loading for you? by calvinf on Wednesday November 17, 2010 @01:18PM (#1915099)
It took this page about a minute to load for me and the images are still loading a couple minutes later.
Re:Ask HN: Has HN been slow/spotty loading for you? by static47 on Wednesday November 17, 2010 @04:43PM (#1915869)
Nope I am having the same issues, glad to know it's not just me!.. although it seems to vary on the time of day.
Re:Ask HN: Has HN been slow/spotty loading for you? by jokermatt999 on Wednesday November 17, 2010 @12:51PM (#1914995)
Yes, and I've checked that it's not the connection or the browser. I'm on XP, so it's likely not the OS either.
Re:Ask HN: Has HN been slow/spotty loading for you? by lukeqsee on Wednesday November 17, 2010 @12:55PM (#1915009)
I've tried it on OS X with Chrome & Safari + iOS. Same results.
Re:Ask HN: Has HN been slow/spotty loading for you? by SriniK on Wednesday November 17, 2010 @12:40PM (#1914951)
Are you having issues with iphone/ipad? Sometimes I too have problem with HN site on my iphone.
Re:Ask HN: Has HN been slow/spotty loading for you? by donniefitz2 on Wednesday November 17, 2010 @02:16PM (#1915342)
Yes, it's been pretty slow lately. The other day it wouldn't load at all for a brief time.
Re:Ask HN: Has HN been slow/spotty loading for you? by kapitalx on Wednesday November 17, 2010 @04:13PM (#1915769)
For me it took 1.5 minutes to submit a comment. Lets see how long this one takes! :)
Re:Ask HN: Has HN been slow/spotty loading for you? by d_c on Wednesday November 17, 2010 @01:19PM (#1915108)
Definitely slower than usual.
Re:Ask HN: Has HN been slow/spotty loading for you? by mikerg87 on Wednesday November 17, 2010 @04:19PM (#1915795)
reading from iPhone is hit and miss. it seems to tell safari on iphone to use older page content. iPad at same point in time always seems to get current stories.
Re:Ask HN: Has HN been slow/spotty loading for you? by abrudtkuhl on Wednesday November 17, 2010 @12:58PM (#1915021)
Yea definitely slower
Re:Ask HN: Has HN been slow/spotty loading for you? by xentronium on Wednesday November 17, 2010 @01:30PM (#1915153)
Okay for me, though, I am in GMT+3.
Re:Ask HN: Has HN been slow/spotty loading for you? by Semiapies on Wednesday November 17, 2010 @04:30PM (#1915833)
It's not just you.
Re:Ask HN: Has HN been slow/spotty loading for you? by sabat on Wednesday November 17, 2010 @01:44PM (#1915214)
Posting seems slower than it used to be. I'm sure PG is on the case. No complaints. I'm glad HN exists.
Re:Ask HN: Has HN been slow/spotty loading for you? by EtienneJohnred on Wednesday November 17, 2010 @03:12PM (#1915538)
If I just answer "yes" without a long dissertation, will I get downvoted by the elite here? My guess is yes.