News for hackers. Stuff that matters.

A comment on Wikipedia's robots.txt. Parent comment

Re:Wikipedia's robots.txt by nkozyra on Tuesday July 16, 2019 @08:20PM (#20455216)
I don't filter out anyone but bad actors. If you abide by robots.txt you're free to scrape my sites
[ Parent | on HN ]

There are no comments on this comment yet.