No, I took screenshots to ensure it.

Your skepticism is warranted though - I was a part of an AI safety fellowship last year and our project was creating a benchmark for how good AI models are at geolocation from images. [This is where my Geoguessr obsession started!]

Our first run showed results that seemed way too good; even the bad open source models were nailing some difficult locations, and at small resolutions too.

It turned out that the pipeline we were using to get images was including location data in the filename, and the models were using that information. Oops.

The models have improved very quickly since then. I assume the added reasoning is a major factor.

A) o3 is remarkably good, better than benchmarks seem to indicate in many circumstances

B) it definitely cheats when it can — see this chat where it cheated by extracting EXIF data and wasn’t ashamed when I complained about it cheating: https://chatgpt.com/share/6802e229-c6a0-800f-898a-44171a0c7d...

As a further test, I dropped the street view marker on a random point in the US, near Wichita, Kansas, here's the image:

https://cdn.jsdelivr.net/gh/sampatt/media@main/posts/2025-04...

I fed it o3, here's the response:

https://cdn.jsdelivr.net/gh/sampatt/media@main/posts/2025-04...

Nailed it.

There's no metadata there, and the reasoning it outputs makes perfect sense. I have no doubt it'll be tricky when it can be, but I can't see a way for it to cheat here.

This is right by where I grew up and the broadcast tower and turnpike sign were the first two things I noticed too, but the ability to realize it was the East side instead of the West side because the tower platforms are lower is impressive.

Oh hey Tyler, nice to see you on HN :)

Yeah it's an impressive result.