I think much of the distrust stems from: "why is google doing this". It costs some amount of money to operate a DNS service, and it's pretty reasonable to think that Google is getting something in return. Pretty much every other service they offer for "free" pays for itself by monetizing usage patterns and content, so it's no leap at all that they are offering DNS services with a similar quid-pro-quo.

Even if they don't use your traffic history to help with personalized advertising, they could conceivably use it for other things (e.g., bot detection, usage stats).

Google discusses what they collect in their privacy policy (https://developers.google.com/speed/public-dns/privacy), but not how they use it.