News for hackers. Stuff that matters.

A comment on Reddit has updated its robots.txt to block all web crawlers.

Re:Reddit has updated its robots.txt to block all web crawlers by seeknotfind on Thursday July 04, 2024 @12:00PM (#40875895)
All of Reddit was freely and readily available just a few years ago. Just goes to show - archive and save what you love.
[ Parent | on HN ]

There are no comments on this comment yet.