I never really used the Google contacts feature until getting an Android phone. At that point I had to spend an hour getting my contacts into a usable state; most of that time was spent clearing out nearly six years' worth of contacts that were auto-added courtesy of Gmail.

What they should do is to allow you to remove everything that you haven't replied back to.

That should be something you can hack with the Contacts Data API:

http://code.google.com/apis/contacts/docs/3.0/developers_gui...