News for hackers. Stuff that matters.

A comment on By removing “extremist content,” platforms are purging human rights evidence. Parent comment

Re:By removing “extremist content,” platforms are purging hu... by TJSomething on Wednesday October 23, 2019 @07:41PM (#21340199)
This policy was actually changed in 2017 to completely ignore robots.txt.

[0] https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...

[ Parent | on HN ]

There are no comments on this comment yet.