Maybe a more subversive method would be publish the page with gibberish, initially, then republish the real content a day after?

I’ve actually done something like this in the past. I was able to identify the IP address of a few content scraping websites, and then made my RSS feed return a huge number of gibberish pages when requested from these IPs. I’m not sure whether they eventually ran out of database space or what happen, but these sites did go down after a few days and stayed down. https://www.ctrl.blog/entry/defacing-content-scrapers

(I didn’t attack these servers, by the by. They came to my servers and gobbled up all the auto-generated junk I served them all on their own.)

However, in recent years it has been more and more difficult to identify the right IP address as everything is hosted people are hosting behind Cloudflare or do the actual scraping from a short-term lease server with an unique IP.

Actually, shortly after I learned that Google switched to neural networks for search results, I noticed an old style of spam making a huge comeback: Either markov chain or neural network generated text, taking from related websites (I suspect they look at the top 10 websites for certain valuable terms).

This gibberish actually outranks legit content which refers to my content, sometimes even my own articles, especially when it is turned into a PDF.

Seems like it is easy to block ~250k webpages like:

https://www.google.com/search?q=inurl:?yhjhuyfib

but I guess Google keeps them there to keep the spammers in the dark? I hope so, else their new ranking signals allow for easier spam.

> Seems like it is easy to block ~250k webpages like

I think it's unfair to look at these results and say "but it's so easy to block these". Google's time is best spent on solutions which will reliably and automatically block these, without going through fairly manual steps.

I did notice a decrease in quality in gmail's spam filter though. Increase in false positives and false negatives lately. I guess it's unlikely to be related...

To me it seems all the work of the same spammer(s). In such a case, do some manual intelligence and wrap it up. It won't scale to all forms of spam, but if a simple regex can uncover 250k+ results in 10 minutes, a manual spam fighter can still block millions of pages a day (and warn the webhost, remove these flakey ads from their networks, etc.).

No doubt the recent machine learning hype has given spammers more advanced tools to avoid detection.

False positives are far more problematic than false negatives...

If you remove from index... sure. But for that URL that I posted, do you think there is even a single false positive in there?

That would probably work, but it would affect legitimate users. They would see the gibberish, too.

You're right, no plan is perfect. I guess it might work if most readers are not going to the page as soon as it's first published though?