Re:Internet Archive as a default host-of-record for startups
by pseudalopex on Sunday January 02, 2022 @08:32PM
(#29776275)
Respecting robots.txt would mean not saving pages blocked by robots.txt. Not just hiding.
[ Parent
| on HN ]
There are no comments on this comment yet.