Re:Wayback machine gets a facelift, new features
by mmagin on Tuesday November 05, 2013 @04:45AM
(#6674218)
Actually, if you want to interpret robots.txt that way, it raises the problem of "how long can I consider a robots.txt valid for?"
[ Parent
| on HN ]
There are no comments on this comment yet.