News for hackers. Stuff that matters.
Re:Wayback machine gets a facelift, new features by Cameron_D on Monday November 04, 2013 @07:16PM (#6672157)
The worst part is when a site is stored in the wayback, then the domain expires and the new owner (or squatter) has a robots.txt that blocks everything, then all the old content becomes inaccessible.
Re:Wayback machine gets a facelift, new features by daveid on Tuesday November 05, 2013 @03:21AM (#6673890)
They should store a history of WHOIS data for the site's domain and make separate archives for when the owner changes, I think. Also, why did anyone think that applying robots.txt retroactively is a good idea? :/