For bot detection, when was the golden age? I'd say that, rather than follow the above description, it started as non-existent and there was basically a smooth descent downward to where we are now.
Sites with people hired to optimize the robots.txt file are ranked higher than those without. If we ban all websites that hire people to optimize their robots.txt, then a lot of the web isn't indexed and the search results suffer.
Or if we ban every creator on tik tok who posts a lot of text in a short video, then a lot of creators are going to be de-platformed and moved elsewhere.
The point of what I wrote is that 'humans are really god at following rules, so much so that they often are able to manipulate rules to their advantage, without breaking said rules.
That's why this is such a hard problem to counter.
Their only job is to stack order the internet. Down ranking a site to the 1,000,000th result is the same as banning them. They necessarily do this all the time (like for every search with more than 10 potential hits).
Perhaps there is no border to be defined since nearly all SEO is abuse?!
The only acceptable SEO should be "provide website content that is more relevant/interesting for the viewer".
And where is the line between "provide website content that is more relevant/interesting for the viewer" and SEO? If I realise that (to take an example from something found on Google) I could take the sentence "Identify the best customers and convert more" and rewrite it as "Marketing automation helps you identify the best customers and convert more", is that SEO because I'm intentionally adding a keyword, or is that making the content more relevant to the user by being more explicit in what I'm saying?
This does of course not imply that these things can be proven, which implies that no action can be taken. Not everything that is immoral is punishable.
1) New system comes out that indexes/controls/regulates a naively created dataset
2) Data consumers adopt that system and experience benefits
3) Data suppliers learn rules of system and take advantage of it to improve the positioning of their data, thus breaking the intent of a system built on the assumption of naive creation
4) Users complain about the broken system
5) New entrants realize that the original system actually solved the core problem really well, and there are no easy ways to solve the 'gaming the system' problem
6) Flawed system remains the best available option indefinitely
It's like entropy, there's just no fighting it.
EDIT: And to extend this beyond Google - do you see a lot of long text blocks in 7 second tik tok videos? That's because the creator found a way to game the algorithm.