That's not to say that there haven't been problems with ads, gratuitously bloated JavaScript, etc. but it's easy to forget that back in the 90s there were a ton of pages which only worked well on then-fast (T1+) connections or relied on Java or Shockwave (not Flash, the Macromedia plugin before Adobe acquired them) and took a solid 30 seconds of swapping just to start up.
The one thing we've definitely seen a big regression on were pages which cannot be used if any part of a huge amount of JavaScript fails, and I'm really glad that mobile devices have been reminding the entire industry why progressive enhancement is still critically important.
Anyways, I do agree that the 90s web was nicer in many ways. One thing I miss greatly is that people actually hand-crafted websites back then. Sure, they used frames and had construction gifs... but they were hand-crafted. If you searched for "star wars" you got a couple dozen sites that were all different looking (but had similar content). Today this is virtually unheard of. Go try to find a star wars fan website that doesn't use Wikia or some other similar tool.
Also why would a news website care if you have a shitty experience. Their A/B tests are saying customers are more likely to click on ads and sign up if they use the dark patterns.
Their end goal is to make more money. That's what they are optimizing for. If you don't like it, don't go to the website. That's the only signal they give a shit about.
Speaking as someone who's worked on browser engines for years now, this is a hard problem. The semantics of the Web demands immediate, synchronous reflows: after performing DOM mutations, any script running on the page can ask for the location of any element on the page, and it requires an up-to-date answer. These APIs are used a lot: almost any news site, for example, is going to use them during initial page load (e.g. for ad placement). So we have to reflow during initial page load.
You might then ask: Why not perform the reflow "in the background" so that the DOM APIs doesn't break, but present a static version of the page as it was initially to the user? The problem with that is that clicking on links (or even mousing over them) is a DOM event that script can be involved in. Besides the fact that you'd basically have to have two simultaneous versions of the DOM around, you'd (for example) break scripts that expect the X/Y coordinates of a mouse-down event on a button to occur inside the boundaries of the button.
Browsers have been coalescing reflows as much as possible within the constraints of the Web, and have for at least a decade. But the semantics of the Web limit what can be done.
However, I'm fairly certain that they do it on purpose to increase the click-throughs to ads. Sites like Salon on my phone are the worst. No matter how much time has passed since it looks like everything has finished loading, the moment I go to click on a link an ad pops up under my thumb.
But Sturgeon's law being what it is, we're probably stuck with a shitty web. But I wonder if browser makers can do anything to ease the pain a bit. For example, a page will reflow a few times while it loads, making it hard to start reading immediately. I'll often be mousing to the thing I want to click on next only to see everything move. Could browsers coalesce these reflows so that only completed pages are presented to the user?