You're right, it's not the only way. That's why I leave only location history on and turn off everything else. I get weather, appointments, stocks and directions to home.
Lovely.
WiFi network + frequent location can probably put you at your house, your office, or your bus stop almost instantaneously (with corrections coming as GPS updates), meaning that using location-based services like Find my iPhone, map routing, or (my most frequent use case) nearest bus stops and their times, can be accomplished much faster and with less battery life.
Likewise, a poor GPS location + frequent locations can narrow you down much faster even with WiFi turned off.
As long as this data is stored only on my phone and not provided wholesale to applications, I'm fine with it; otherwise, it's concerning.
Or it could ask me to identify locations that I care to have context-aware functionality for -- and then not build a complete history of my movements, with all the associated privacy risks involved, just so that it can guess at what's important to me.
This is my core complaint with Google Now as well. This "track everything and try to sift meaning out of the data" approach is not the only way. It requires someone amass have a massive pile of data on me that I am flatly uncomfortable with.
There's the concern about corporate data sharing/selling, corporate data breaches and now the NSA. It's flatly Just Not Worth It, just so that my phone can spare me the trouble of manually entering a handful of addresses. (Most of the addresses I care about are already in my contacts list.)
That telcos have this information is itself a fault in the system. Not an excuse to give it to everyone.