Re:Internet Archive as a default host-of-record for startups
by kodah on Tuesday December 21, 2021 @03:40PM
(#29641844)
I mean even honoring the opt-out of robots.txt would be fantastic. As another commenter pointed out they willfully ignore it: https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...
It's fairly unethical.
1.) Most people won't opt-in because a significant majority accept defaults and don't opt into most things.
2.) For people like yourself probing a bit deeper, you might well ask whether you really want to give up your ability to decide you don't want something you thought was so funny when you wrote it at 20 now that you're a politician running for office or up for a political appointment.