News for hackers. Stuff that matters.
Re:Reddit has updated its robots.txt to block all web crawlers by The_Colonel on Thursday July 04, 2024 @09:14AM (#40874691)
I wonder how much they pay. Reddit profits a lot from showing up on the top for many search queries. I very often do "whatever I'm looking for reddit" (for e.g. product reviews), since the reddit results often provide higher quality information than normal results.
Re:Reddit has updated its robots.txt to block all web crawlers by toomuchtodo on Thursday July 04, 2024 @11:08AM (#40875519)