HTTP directly says you should use authorization to make stuff non-public.

And third-party content scraping tools are NOT violating copyright law, as they do not redistribute it freely.

Otherwise services like Opera Turbo – which scrapes a website, removes cruft, compresses data, and sends it to your phone – would also break copyright law, and there is a nice exception for this case, under US law, it’s covered under fair use.

(Unless the scraping service makes your content accessible publicly – and even then, fair use applies usually, as in the case of pure archival like archive.org or archive.is)

Are you a lawyer? I am not - anyway, I can't really agree with that falls under fair use.

If I charge money to scrape someones site and then hand over the material in whatever form my customer uses - I am bound run into problems sooner or later.