News for hackers. Stuff that matters.

A comment on If you love a website, back it up.

Re:If you love a website, back it up by whartung on Monday November 02, 2026 @02:22PM (#50342180)
So, this is a fine idea and all, but aren't the "piss off AI bot" mitigations going to throw a wrench into folks wanting to back up a website?

I guess the web plugins could ideally get around it, but what about the CLI and app solutions?

I know yt-dlp has gone to great lengths to work this problem.

[ Parent | on HN ]
Re:If you love a website, back it up by qlte on Tuesday November 03, 2026 @01:40AM (#50348812)
Client scraping with SingleFile extension or Karakeep works perfectly on anything visible in your browser window from my experience. Including SPAs/JS rendering/aggressive CAPTCHAs/etc.

For CLI automation could probably one of various fingerprinting resistant Chromium instances with the extension setup in a Docker container or similar. Run from your home Internet for the residential IP.

[ Parent | on HN ]