Would they? Is there any precedent here of Apple scraping user info? The usual complaints against Apple (mine included) is that their systems are far too locked down and closed to competition- not that they do funny stuff with my data.
http://www.wired.com/gadgetlab/2011/04/iphone-customers-laws...
http://www.apple.com/ca/pr/library/2011/04/27Apple-Q-A-on-Lo...
All smartphone providers track the same sort of data in roughly the same way, it's part of what makes location software work.
Kinda like HIPPA medical privacy rights in the USA: when you go to a doctor, they'll hand you a paper for you to sign acknowledging you understand your privacy rights...then they hand you another paper for you to sign away those rights; if you don't sign both, you don't get any medical service. Imagine a not-distant future where if you don't opt-in to questionable services from Google, Apple & Microsoft, you're flatly denied use of any of their products - digitally ostracizing yourself from most of society.
And personally, I'm fine with Google doing so when they're actually doing something for my benefit with it (like Now).
People complain here all the time about Google getting access to your personal data.
Guess whether a walled garden is a bad thing depends on your particular brand of fanboyishness.
Personally speaking, the idea of Google displaying things based on my email history, diary and a knowledge of my location makes me shudder, "You're near that motel where you had that quickie last month, want to make a reservation?".