General tip: If HN is being laggy and you're determined you want to waste some time here, open it in a private window. HN works extremely quickly if it doesn't know who you are.

Wow this really works, thank you. What actually is the reason for it being much faster in a private window? Is there so much tracking going on in a normal window?

its faster because the pages are cached, they are effectively static. It's slower when logged in because the pages are created dynamically as it has your username, tracking favourites, upvotes etc, and much of it cannot be quickly cached.

Honestly surprised that HN, a website for techies, is so poorly coded. For example, the whole lack of proper "paging", with dang posting a disclaimer on every large thread for honestly over a year at this point and no progress. Or the fact that if you want to reply to a comment, it has to load a whole new page (which has to fetch more data from the server?) before getting to the comment box. Until recently, trying to collapse a large comment thread would also take 3s+ as I think it individually set the collapse state of every sub-comment?

The whole thing was put together in a somewhat obscure dialect of Lisp over 15 years ago. There’s probably under 100 people that write Arc regularly enough to meaningfully contribute, so the general approach has been to not fix what’s not broken.

This is not a very complex website, any HN reader could probably whip up a replacement from scratch over a weekend.

I guess there does exist many alternative UIs, though I don't see many that support commenting. I wonder if the "API" (if there's any) allows for that, or if people are just scraping the page and reformating it.

Not to argue, just to post a contrasting view: while FB, and a lot of the internet, failed or slowed today—and I know there were tons of reports of HN slowing too—I also experienced a phone death and attempted to hobble along by putting my SIM back in my old iPhone 5. Basically the only thing that worked was HN. In fact it loaded as quickly as I’d expect.

There’s plenty of stuff I’d like to be different about usability of this site, but perf is basically at the bottom of that list.

Most of the things I listed weren't really perf related, though they do show up when there's perf issues. Being able to "load more comments" and reply inline are super basic usability features. There's no reason why I'd need to navigate to a whole different page with a textbox, then navigate back and lose my position every time I post a comment.

One of the first optimizations large/high traffic sites will do, is cache pages for logged out users. even if the cache is only valid for a minute, that's still a huge reduction in server traffic.

The cache is faster because its not having to talk to the database, and can be done at by the load balancing layers rather then the actual application layer.

Wikipedia does this too (although, via a layer to add back on the ip talkpage header).

You can also just log out instead of opening a private window. Users that aren't logged in are served cached pages.

Could they offer cached pages to logged in users as an optimization? You only need to invalidate when a user posts a comment, most of the time you are reading now commenting?

This would make for a very good deep-dive technical discussion in an interview setting, I'm using this.

That explains why it works fine in my computer, where I haven’t logged in. Thanks for the tip.

This works like charm. Thank you!

I can confirm, HN, GitHub and Slack are very slow for me as well. Google is very fast, on the other hand.

All running their DNS on AWS. My guess is that AWS is seeing a massive flood of failed and retried DNS requests for facebook properties, similar to what jgrahamc mentions here for Cloudflare: https://twitter.com/jgrahamc/status/1445066136547217413

Is there a "Kessler syndrome" analogue for the internet, where failures beget failures until it's just an impenetrable cloud of fail, forever?

There's such a thing called the "Thundering Herd" problem, that partially matches.

From wiki: the thundering herd problem occurs when a large number of processes or threads waiting for an event are awoken when that event occurs, but only one process is able to handle the event. When the processes wake up, they will each try to handle the event, but only one will win.

Until someone smashes the "SEND MOAR SERVERS" button.

I can't see how this is the reason for HN to take 10 seconds for the response of the main page (I mean, the URL fetched from the address bar, not the subrequests the page does), as everything else downloads immediately.

The DNS entries should be cached by the browser (and the middleware), so that this problem should only happen once, but I get this constantly.

Also, I sometimes get an error message from HN, which seems to indicate that this is some backend issue which fails gracefully with a custom "We're having some trouble serving your request. Sorry!" on top of a 502 code.

It feels more like there is something else still broken.

In the case of HN it's probably just heavier load than normal. It's much faster if you're logged out.

Dropping that many BGP routes will have its high latency toll on the whole internet backbone for minutes/hours, I'm not surprised. I wonder if the recent LE's DST Root CA X3 deprecation has something to do with the outage (some DC internal tool/API not accessible because its certificate is expired or something like that).

Also slow here. I can't see anything on the AWS Service Dashboard https://status.aws.amazon.com

In my experience, any service dashboard is useless unless the problem has been going on for so long (i.e. hours) that it is obvious something's wrong.

AWS punishes its sysadmin teams for any downtime so there is heavy incentive to not report unless there is a community shaped gun pointed at your head. This is not a universal problem.

AWS punishes its sysadmin teams for any downtime so there is heavy incentive to not report unless there os a community shaped gun pointed at your head. This is not a universal problem.

People either have to work, creating load on GitHub, or waste their time elsewhere, creating load on HN and Slack.

People probably got more time to work.

Also slow for me also.

Probably traffic related. Lots of people reallocated to checking other sites.

Probably people flooding in to see if anyone knows why things are down. Even Google speed test was down, presumably from too many people testing if it's their internet that's at issue.

https://www.speedtest.net is down too

The site is working fine for me. Speedtest CLI also is useful but doubt when DNS is down.

working for me

I'd guess that automatic processes dominate. Maybe billions of phone apps polling for facebook connectivity (FB messenger is down, for example).

A couple of years ago, an admin at Hacker News asked those of us who are just reading to log out because their system is architected in such a way that logged in users use more resources than anonymous ones. So, if you're feeling altruistic, log out of HN!

Logging out does work! Probably delivering a cache.

Internet's got a case of the Mondays for sure

Yep. I am the developer of HN client HACK for iOS and Android and a bunch of users emailed me asking if my app was broken. Looks like something bigger is afoot.

Something's wrong with your app. It's not working at all, while Harmonic works perfectly.

Harmonic most likely uses Algolia for the data whereas my app uses the HN website. So Algolia delivers a cache copy from their own servers whereas mine scrapes the HN website itself. Hence the difference. Also logged out pages were working much better than logged in (HN delivers caches copies for logged out users).

Best HN client app ever. Thanks for the great work!

Thank you!

I can confirm, HN, GitHub and Slack are very slow for me as well. Google is very fast, on the other hand.

EDIT: actually HN failed to post this comment the first time I posted it!

Not just you. It is very laggy on my end too.

HN lagging, BBC was also very laggy about 30 minutes ago, and 35 minutes ago our whole company got booted out of their various hangouts simultaneously apart from the people in the states.

Definitely laggy for me as well. Went to Facebook and couldn’t so come here to check in and the load time made me think oh it must be my wifi is not working with 2 sites not opening then finally HN opened. Then tried to hit reply to your post and again seemed like it wouldn’t load then finally did. So yes laggy usually this is the one site that loads almost instantly.

Here too. Just had the "We're having some trouble serving your request. Sorry!" error.

With the Facebook properties down, the rest of the internet will have a significant increase in usage.

Plus I don't know about you, but I came to HN just now specifically to check if there was any insight into why it was down! The thundering herd just arrived :)

Yes, it's slow here as well, and posting this comment failed the first and second time.

Can confirm. HN, YT, Google, etc are all a bit laggy for me at the moment (eating lunch so I'm trying to entertain myself).

Yes, it's slow here as well, and posting this comment failed the first and second and third and fourth time.

This is either a hilarious accident or genius comedy.

This is not too rare when HN is being slow and giving those "We're having some trouble serving your request. Sorry!" pages.

If you get one of those on your comment submission you have no way to know if the trouble stopped it from accepting the comment or if it accepted the comment and ran into trouble then trying to display the updated thread.

For some reason I can't even begin to guess at HN does not seem to have protection against multiple submissions of the same form, so if after getting "We're having some trouble serving your request. Sorry!" on your comment submission you hit refresh again to display the page and the form gets resubmitted, you get a duplicate comment.

Earlier today when I was getting these I went to go check the page to see if the comment was posted there. More than once it said it failed but I was able to stop myself from trying again because it was actually there.

Yes, it's slow here as well, and posting this comment failed the first and second and third time.

Yes, it's slow here as well, and posting this comment failed the first time.

It is slow for me too.

Same here. Sounds like another cloudflare-like problem.

yes, me too (I'm in south-america)

Yep, struggled to load the homepage and this

yeah lagging for me too

Some internet backbone provider is probably down itself.

Or some country has started a war.