News for hackers. Stuff that matters.

A comment on Only Google is really allowed to crawl the web. Parent comment

Re:Only Google is really allowed to crawl the web by WesolyKubeczek on Friday March 26, 2021 @12:45PM (#26594198)
I don't scrape a website often, but when I do, I'm using a user agent of a major browser.
[ Parent | on HN ]

There are no comments on this comment yet.