Re:Only Google is really allowed to crawl the web
by Spivak on Friday March 26, 2021 @02:02PM
(#26595042)
Spoofing user-agents hasn't worked in a long time for anything but small operations because search engines publish specific IP ranges their scrapers use.
[ Parent
| on HN ]
There are no comments on this comment yet.