Especially at the position of Google, it's neither a coincidence nor something taking them by surprise. They've been in an arms race with spammers and SEO gurus since the beginning, and had all resources on earth to deal with it. The results being bad as they reached a monopoly position is of their own making, or more precisely a situation they saw as the best tradeoff.
That's where I don't see Google turning the ship around: people who were there for decades to make searches relevant have probably already left a long time ago, or they accepted the new direction themselves. And new people with expertise and enough energy to rock the boat are also in a position to do more impactful work in other companies where they won't have to fight an establishment.
As a example of how hard it is to change direction, we haven't seen Microsoft suddenly build customer friendly OSes, nor Meta having good taste, nor Apple come up with open ecosystems.
The rule should be that only good faith web sites are allowed. Web sites with dark patterns are not allowed. Anyone who is doing any SEO shenanigans gets their entire domain de-indexed permanently. If Google does this, and only this, it will return wonderful useful search results.
We might be completely mistaken on the reasons why it's hard, but there's no doubt they can't do it easily.
On human curation, this brings other problems that Google is also very bad at solving. Google would need to manage their moderators at scale. Even Meta couldn't do it in a sustainable way, and Amazon's Mechanical Turk model is also deeply problematic. Amazon itself puts a ton of effort into customer service and yet doesn't keep up with the scams.
Curation and moderation at Google scale is indeed a crazy hard problem.
That's probably the only reason we're getting the truely creative stuff that was thrown in with no expectation of monetization.
Otherwise the useful bits look decently surfaced to me. Albeit not in a searchable or convenient way.
But Google has one resource that others do not, which genuinely contains a lot of good content. And that is YouTube. If they could simply make the content of videos easier to search, they would be able to offer a unique set of useful answers. At least for now before AI generated garbage takes over YouTube.