Although, I agree with you in principle. In reality it is a little different. Just because you don't take your phone out doesn't mean other people aren't. Its the same principle as Facebook being able to figure out who your relatives are even though you may not be on Facebook. Its all about data points. With enough data points, it doesn't matter whether you bring out a phone or not.