It's kind of a matching and scaling problem. The ad publisher wants to have a well targeted audience that will convert well, while the website wants to generate ad revenue that won't drive people away.
The problem is that the ad publisher probably doesn't want to deal with the hassle of negotiating with dozens of niche websites for their audience, while the websites don't really want to do the same either. Hence, the rise of ad exchanges, which offers benefits for the ad publishers (deal with only a handful of exchanges, all kind of ad targeting criteria) and the websites (deal with only the exchange, add an ad snippet to generate revenue almost immediately).
The catch is that to extract the most money from ads, the exchanges do all kinds of tracking to figure out the most optimal ads to serve and we end up with the current adtech industry, instead of manually targeted static ads.
> we count the number of uBlock Origin blocked requests on the page, and if too many, we kick it out, leaving only "clean" pages in the index.
And I get that people make a living by putting advertising on their pages. I'm not really against that. (Though I don't do that myself for my own reasons, I wouldn't presume to tell other people what to do with their websites.) But it's been abused to the point where number of ads on a page is a decent proxy for how crappy the content is.
When people decide to put ads on their content, the temptation is there to produce content to carry the ads. Before ads in the old days, you just produced content to produce content. Again, I'm not saying that ads are bad. But the purity of the old web is lost.
I don't think there's really a way to bring it back. I think the answer is that we just have to get better filtering on our ends to get the specific information that we want.