News for hackers. Stuff that matters.
Re:Robots.txt is a suicide note by marshray on Tuesday May 10, 2011 @01:36AM (#2531487)
however the de facto modern use for it goes far beyond that, and ignoring that standard is likely to piss off lots of people

I think they're saying is that their archive is going to ignore it, and your website should just eliminate it.

We'll have to assume that archiveteam.org are by now real experts in guessing which buttons are going to be pissing people off. :-)

There are no comments on this comment yet.