Re:Wayback machine gets a facelift, new features
by derekp7 on Monday November 04, 2013 @08:45PM
(#6672649)
Really? I was always taught that search engines only do "get" requests, and anything that modifies data is in a "post" request. Are there really that many broken web sites out there, that hasn't already fallen victim to crawlers that ignore robots.txt?
[ Parent
| on HN ]
Re:Wayback machine gets a facelift, new features
by greglindahl on Monday November 04, 2013 @09:17PM
(#6672773)
Yes, there are a lot of broken websites out there.