if you are logged in, it has to check if you upvoted some of those listed items (submission and comments). If you're logged out, it doesn't need to check anything - so it's faster

In fact, if I remember right, when logged out it can serve cached, pre-rendered pages. Sometimes when HN is down or underperforming, clearing your cookies or opening in incognito will still allow you to view the site because the cache is still present.

I'm always logged in so I haven't got a good impression of the difference but it seems lightning fast most of the time when logged in, so it can't be too much of a difference I guess.