Re:Reddit has updated its robots.txt to block all web crawlers
by The_Colonel on Thursday July 04, 2024 @09:14AM
(#40874691)
I wonder how much they pay. Reddit profits a lot from showing up on the top for many search queries. I very often do "whatever I'm looking for reddit" (for e.g. product reviews), since the reddit results often provide higher quality information than normal results.
[ Parent
| on HN ]
Re:Reddit has updated its robots.txt to block all web crawlers
by toomuchtodo on Thursday July 04, 2024 @11:08AM
(#40875519)