Re:Ban me at the IP level if you don't like me
by dilyevsky on Tuesday August 26, 2025 @03:04PM
(#45030793)
I thought the argument was that if you run on gcp you can masquerade as googlebot and not get a 429 which is obviously false. Instead it looks like the argument is more of a tinfoil hat variety.
btw you don't get dropped if you issue temporary 429s only when it's consistent and/or the site is broken. that is well documented. and wtf else are they supposed to do if you don't allow to crawl it and it goes stale?
The point is they’re getting paid to run cloud servers to keep their bots happy and not dropping your website to page six.