News for hackers. Stuff that matters.

A comment on Help Us Keep the Archive Free, Accessible, and Private. Parent comment

Re:Help Us Keep the Archive Free, Accessible, and Private by CM30 on Thursday December 01, 2016 @06:06AM (#13078624)
Why not look at WHOIS information when getting an update, and then class a site as 'different' based on whether that changes? In most cases, a new domain owner usually means the site isn't the same as the earlier versions.

You'd then just have to stop the archive indexing/showing content after the WHOIS information changed, while leaving the stuff before it intact. Maybe you'd then have a nice form to report pages you want removed/hidden (for the edge cases), or even a seperate robots.txt/meta declaration you can make confirming you're the same person that owns the site. After all, most of the reasons why sites go missing aren't deliberate attempts to rewrite history, but domain squatters not wanting holding pages indexed.

Feels like it'd be so easy to implement robots.txt in a more logical way on the Internet Archive.

[ Parent | on HN ]
Re:Help Us Keep the Archive Free, Accessible, and Private by db48x on Saturday December 03, 2016 @01:26AM (#13094631)
It's been suggested, but there's no way to automatically do it correctly. The whois info might be anonymized, in which case a change means nothing at all. It might just be someone's name and address, with no way of verifying who that someone works for. Also meaningless. Better just to default to something safe, and spend your manpower on something more important.
[ Parent | on HN ]