Title says o3 beat a [human] player. That implies there is some competition that has the capacity to be fair or unfair.

Presumably being an AI is technically against the GeoGuessr rules so I don't see how there can be an expectation that those rules were followed.

Sure, but o3 is itself already an online service backed by an enormous data set, so regardless of whether it also searched the web, it's clearly not literally "playing fair" against a human.

But it still bounds the competition. OP is skilled in the domain. I'm not, so if I wrote a post about how O3 beat me you'd be saying how mundane of a result it is. I mean, I suck at Geoguesser. Beating me isn't impressive. This is also a bound

> But it still bounds the competition.

Bounded to...a model trained on virtually all publicly available text ever generated by humans. I wouldn't expect web searches to even help much unless they're turning up data from after the model was trained.

  > Bounded to...a model trained on virtually all publicly available text ever generated by humans

Don't forget there's a lot of non-public data too!

I don't disagree, but my point is that some bound is better than no bound. I think we can certainly agree that there are even better bounds than others. Obviously we won't ever have a full equal comparison, but I think the bounds do allow for some insights to be gained. We just need to be cautious that those insights consider those bounds (I believe we both are cautious about what insights can be gained. If you doubt me, see my other comments. I do push back on OP pretty hard)

  > unless they're turning up data from after the model was trained.

Only under the condition that the models perform lossless compression of all data trained on. If the compression is lossy, then search will reduce that loss.