The problem we have with a lot of modern technology, and particularly communications and payment technologies, is that while some degree of personal identification and association with specific locations or actions is necessary to perform their function, that same information is readily repurposed and easily shared if there are no rules to prevent it, or almost equivalently, if there are no effective sanctions if rules that do exist are breached.
For a single tower, yes. In practice, if you're within range of multiple cell towers, a little physics and mathematics will determine your location much more precisely, assuming knowledge of each tower's exact location, the relative signal strength received from your device at each tower, and ideally any confounding factors like awkward terrain or local sources of interference.
As an aside, if things are going wrong on the network, it's a fair bet that engineers will be out driving around with directed antennae that will find any given rogue transmitter even using a single receiver given a bit of time. This one's not so much of a privacy concern, though, given that it only tends to be used as necessary to resolve specific problems.