Re:Only Google is really allowed to crawl the web
by MinorTom on Friday March 26, 2021 @08:10PM
(#26598503)
CGNAT
[ Parent
| on HN ]
Re:Only Google is really allowed to crawl the web
by d110af5ccf on Sunday March 28, 2021 @02:09AM
(#26608373)
Now that's a really good point. I wonder why there isn't a standard protocol for signalling upstream that a particular connection is abusive and to please rate limit the path at the source on your behalf? It would certainly add complexity, but the current situation is hardly better.