I've run a web-scale search engine and I don't think it will work.

Not only are some sites malicious -- mostly unimportant ones -- but many good sites are simply incompetent.