The rule should be that only good faith web sites are allowed. Web sites with dark patterns are not allowed. Anyone who is doing any SEO shenanigans gets their entire domain de-indexed permanently. If Google does this, and only this, it will return wonderful useful search results.
We might be completely mistaken on the reasons why it's hard, but there's no doubt they can't do it easily.
On human curation, this brings other problems that Google is also very bad at solving. Google would need to manage their moderators at scale. Even Meta couldn't do it in a sustainable way, and Amazon's Mechanical Turk model is also deeply problematic. Amazon itself puts a ton of effort into customer service and yet doesn't keep up with the scams.
Curation and moderation at Google scale is indeed a crazy hard problem.
Especially at the position of Google, it's neither a coincidence nor something taking them by surprise. They've been in an arms race with spammers and SEO gurus since the beginning, and had all resources on earth to deal with it. The results being bad as they reached a monopoly position is of their own making, or more precisely a situation they saw as the best tradeoff.
That's where I don't see Google turning the ship around: people who were there for decades to make searches relevant have probably already left a long time ago, or they accepted the new direction themselves. And new people with expertise and enough energy to rock the boat are also in a position to do more impactful work in other companies where they won't have to fight an establishment.
As a example of how hard it is to change direction, we haven't seen Microsoft suddenly build customer friendly OSes, nor Meta having good taste, nor Apple come up with open ecosystems.