I doubt many website owners feel they have an ethical responsibility to provide public APIs to their data so people wouldn't scrape their content. It is up to them what they want to do with it.

I just feel like the whole thing is a chicken and egg problem - both sides could be nicer to the other but no one wants to go first!

I do like the idea of identifying web scrapers in the user agent but I'm not sure how many websites would use it...