My first job out of college involved a Rails codebase littered with conditionally rendered jQuery snippets. It was a nightmare to figure out what code was even loaded on any given page. Give me a modern JS codebase over that any day.
Might it be possible that those expectations, and those requirements for doing "more", are not coming from the actual users/visitors to the websites - but rather from the companies owning the websites?
Because I, as a visitor of HN, come here for the IT news and the nerd stuff. I happily put up with the layout - because the sites actually meets my expectations for content. At the same time, the best, most modern, snazziest website ever... why would I go there (more than once), if there's no content of interest to me?
And that - to my experience - holds true for almost ALL visitors. They come for the content, and they put up with the layout. And this table-based layout with spacer gifs right here, imho is a lot easier to put up with, than some desing-heavy website that's perfectly optimized for conversion, engagement and retention rates and whatnot.
I think the actual problem is, that when we talk about "expectations" and "requirements" - we talk about what the HOST/SELLER wants - not what is best for the visitor.
I absolutely loath frontend developments in companies with more than 5 engineers. In the past you may have found some spaghetti code and various abuses of jQuery but you could probably grasp the codebase in an afternoon. Nowadays every mid company codebase is a spaceship. For sure, you'll find a pseudo technical VP of engineering which enable whatever cool technology to be used and resume driven developers happy to pile on the latest Big Tech framework - which is likely to be an over-engineered, over complicated exercise in engineer retention and engineer roles marketing.
Then you have Pieter Levels pulling a mil per year with PHP and jQuery.