The IP -> location mapping is googles secret sauce, but it collects data from a lot of places, and is for most users very accurate. We're talking street level accuracy in many cases. The data you or another user feeds into it you could easily get back as your reported location, especially if you were trying to fool it.
Your phone doing a wifi scan for nearby access points to determine it's own location, then doing a google search could easily mark your whole home external IP as being in that place (after a few hours), which would then affect non-signed in google services from other devices in your home.
There are cases it's very bad too though, especially VPN's and frequently-shifting dynamic IP's.
Clear cookies, and you'll see it again.
In Chrome, it also uses the TLS Client Channel ID, which is a persistent unique identifier established between a browser and a server which (on capable platforms) is derived from a key stored in a hardware security module, making it hard to steal. Ie. if you clone the hard drive of a computer, when you use the clone, Google will know you are a suspicious person, even though you have all the right cookies.
Sounds like another reason to limit my personal use of Google's Chrome browser.
Dumb question, but how often do you close your browser though? Mine stays running in the background so it's not often that it gets completely closed.
My point is, I highly doubt my browser usage pattern reflects the way the median user browses the web, and yet even my careful usage wasn't enough to keep Google from correlating enough data about my browsing's attributes (IP, sites visited, location info, logged-in geo-IP), and then expose that cross-bleed in my face.
That aside, to have a setting that doesn't do what it says is disturbing. If that's don't be evil then what's their definition of evil?
A week ago, I moved from my previous home that was close to my work, to a different city about 20 miles away. Since I wasn't sure where all the points of interest are, I've been turning on location on my Android phone and using signed-out Google Maps on my phone.
Yesterday, my work computer's signed-out Google Maps has centered its default view on my new city, on the exact highway interchange next to which I now live.