News for hackers. Stuff that matters.

A comment on Only Google is really allowed to crawl the web. Parent comment

Re:Only Google is really allowed to crawl the web by Mauricebranagh on Friday March 26, 2021 @01:01PM (#26594378)
I have never had that problem running screaming frog on big brand sites apart from one or two times.
[ Parent | on HN ]

There are no comments on this comment yet.