I wouldn't do that. It's better to stay under the radar IMO. They usually won't notice the traffic if it's just for personal reasons. And who says the internet can't be used for automatic retrieval?

I'm not going to pound websites with requests every minute of course but I think this falls under legitimate use. Whether I click the button myself or just schedule a script to do it shouldn't matter so much.

But if you ask you draw attention to it and get their legal dept involved (after all most websites are not run by a single webmaster in their bedroom) who will most likely say no because legal people are hesitant to commit to anything.

But maybe my usecase is different. I just scrape stuff to check if something I want is back in stock, to download my daily pdf newspaper I pay for, archive forum posts I've written, stuff like that. I don't index whole sites.

But yeah I do make sure I don't bombard them with requests, though this is more from a "staying under the radar" point of view. And indeed to avoid triggering stuff like cloudflare.

But if you're scraping to run your own search engine and offer the results to the public the situation is much more complex of course, both technically and legally.