News for hackers. Stuff that matters.

A comment on World War II 'rumor clinics' helped America battle wild gossip. Parent comment

Re:World War II 'rumor clinics' helped America battle wild g... by dredmorbius on Wednesday March 06, 2024 @04:28PM (#39621739)
IA have increasingly ignored robots.txt since the mid-2010s, going on at least eight years now:

<https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...>

<https://blog.archive.org/2016/12/17/robots-txt-gov-mil-websi...>

[ Parent | on HN ]

There are no comments on this comment yet.