News for hackers. Stuff that matters.

A comment on A $2 photo from a California junk shop is the ‘holy grail of Western Americana’. Parent comment

Re:A $2 photo from a California junk shop is the ‘holy grail... by TeMPOraL on Friday October 16, 2015 @01:40PM (#10400840)
There's a really bad problem with archive.org - that they retroactively apply robots.txt files. Simple:

    User-Agent: *
    Disallow: /
And poof, here goes history. Guess how many companies care?
[ Parent | on HN ]
Re:A $2 photo from a California junk shop is the ‘holy grail... by dragonwriter on Friday October 16, 2015 @04:26PM (#10401793)
> Guess how many companies care?

Many. But, more in the way of actively and enthusiastically supporting the ability to wipe the archive this way than, you know, being concerned that it is a problem.

[ Parent | on HN ]