News for hackers. Stuff that matters.

A comment on Robots.txt for the NYT has a specific exclusion for an 1996 news article. Parent comment

Re:Robots.txt for the NYT has a specific exclusion for an 19... by anoncake on Monday March 29, 2021 @06:02AM (#26619501)
<outlandish conspiracy theory>

Google does this to make every tech worker not working for Google, and thus Google's competition, less productive.

</outlandish conspiracy theory>

[ Parent | on HN ]
Re:Robots.txt for the NYT has a specific exclusion for an 19... by felipemnoa on Monday March 29, 2021 @09:29AM (#26621214)
Or maybe the google AI has a funny side. Right now it's probably laughing its ass off reading teMPOral's comment.
[ Parent | on HN ]