Reddit used to struggle with this back in the day. The way they did pagination was to say "show the next page after id postId=xyz123". Since the algorithm kept re-sorting the posts on the front page, the content moved around quite a lot and you'd see the same things re-appearing on page after page, and sometimes post xyz123 was deleted and then everything broke.
It’s one of the reasons that the “I can build XYZ app in a weekend” crowd sounds so silly. I bet Reddit spent dozens of not hundreds of man-months nailing their scrolling behavior. It probably took many iterations of UX, product and engineering to get it right. And even now I bet they have a pile of JIRA tickets buried in their backlog relating to the pagination system.
In my experience there is a non linear relation between the more seemless and “easy” something appears and the amount of work that went into it. Things seem “easy” because the developers poured a pile of fit & finish on the system.
The latter point is relevant to a variety of UX decisions across many products and services. Making it easy, efficient, and pleasant for the user to do what the user wants may not be the overriding concern in the design.
Although Reddit's design decisions are almost certainly to accomodate their much larger scale.
Hacker News being designed to display trees sorted by karma also makes it difficult, you would have to re-sort the entire page to re-rank every new subtree within the current display.
Apparently, this also makes pagination difficult, since pagination is implicitly chronological - what people want from pagination is to easily find the latest items - which karma sort doesn't allow.