Re:If you love a website, back it up
by Dwedit on Monday November 02, 2026 @09:25PM
(#50347205)
Recursive wget isn't going to pass Anubis.
[ Parent
| on HN ]
This. Aggressive AI crawlers have ruined many parts of the web. I guess you have to ask the owner of the site nicely, if they would provide a complete copy of the site for your personal archive.