My counter to this is, without Google's "algorithm" the search results would not be great/find what you want. And in this case websites want to volunteer/agree to index their site for visibility but then you still have to visit the site because conveniently The site summary ends before the content that you want. I don't know how I would find websites unless referred on some website.

I kinda felt the same way about how Netflix started out but I think they probably had a deal to pay a portion of revenue from dvd's that they rented out.

robots.txt