Re:Firefox’s latest Test Pilot: No More 404s
by dragonwriter on Thursday August 04, 2016 @12:56PM
(#12226883)
Doesn't really explain why they have to nuke it, even if it is the current site owner. Respecting robots.txt is one thing, but that just means not spidering and archiving the content that is now there. Deleting already archived material based on later changes to robots.txt is a non-obvious behavior, given the usual understanding of the general meaning of robots.txt.
[ Parent
| on HN ]
Re:Firefox’s latest Test Pilot: No More 404s
by aab0 on Thursday August 04, 2016 @03:56PM
(#12228027)
They're not deleting it, just hiding it from public access. Once the squatter goes away, the content comes back.
Re:Firefox’s latest Test Pilot: No More 404s
by J_Darnley on Thursday August 04, 2016 @09:10PM
(#12229502)
What's the difference? Both make this feature (and more general use of the archive) useless.
The difference is exactly what I said: if they deleted it, it's gone forever. If they hide it, it can come back. I've seen pages I cite disappear for a year or two thanks to scummy squatters - but they came back! It's the difference between being sentenced to execution and to 1 year of prison.