I partially agree with your comment, but it feels like in some/many cases the Google algorithm has large flaws that were not previously there and that (superficially) seem solvable. For example:

* If I search for recipes, the Google algorithm seems to heavily favour recipes with a short novel prepended to them. The “novel” does not have to be particularly relevant to the content - I’ve seen recipes with “novels” that I’m not convinced wasn’t partially algorithmically generated. Every human being I’ve ever spoken to hates this.

* Google will search for what it believes to be alternatives to terms, even when those terms are placed into quotes (which used to prevent this behaviour). This can be very irritating when Google’s alternative terms are incorrect.

Both of these are regressions over earlier behaviour, and don’t seem to have any obvious benefit to them.

Don't know about regressions. Did it maybe just take people some time to figure out that Google Bot is really, really captivated by a riveting story to accompany that pancake recipe?

> a whole new level of the entire world trying to game their search

I think their #1 problem is their product managers trying to do "something" to add said something to their resume and making the product horrible to use in the process

Examples include: - Grid view tab switcher in Chrome Android - Removing dislike count on YouTube

As a xoogler, it was clear when the original search quality folks left and were replaced with people whose goal was to grow the product (and the company) without much care for quality.

Different types of these strategies have been around since before google existed. Hell, even web directories (like early yahoo!) had to deal with spam, early google had to deal with keyword spam. Paid backlinks, spammed comment sections with backlinks, copied content sites as backlink farms, all of this existed at least as early as 2005-ish. For some sites (some of them with legitimate info) the meta keywords tag (even though google stopped using it very early) was the bulk of their size for just this reason.

What has so radically changed the last couple of years to make spam take top spots on the SERP?

This is not 2005 spam. This is just how people are expected to write now. More people think it's normal. By spam being more content-like, and, in response, content getting inspired by spam and getting more spam-like, spam is getting increasingly indistinguishable from content.