To be fair, that data is distributed across many entities. What people worry about is any single company aggregating too much data. Google can get all your searches, email, some purchases (through Wallet), social connections and interests, phone calls, voice mails and location data. Now they could literally record everything you see. I think that's why some people might be reticent about using this.