Terence Tao: Quantitative AI progress needs accurate and transparent evaluation

Re:Terence Tao: Quantitative AI progress needs accurate and ... by sebzim4500 on Friday July 25, 2025 @08:38AM (#44682452)

If every photo in streetview was included in the training data of a multimodal LLM it would be like 99.9999% of the training data/resource costs.

It just isn't plausible that anyone has actually done that. I'm sure some people include a small sample of them, though.

[ Parent | on HN ]

Re:Terence Tao: Quantitative AI progress needs accurate and ... by bluefirebrand on Friday July 25, 2025 @09:07AM (#44682728)

Why would every photo in streetview be required in order to have Geoguessr's dataset in the training data?

[ Parent | on HN ]

Re:Terence Tao: Quantitative AI progress needs accurate and ... by bee_rider on Friday July 25, 2025 @10:59AM (#44683924)

I’m pretty sure they are saying that Geoguessr's just pulls directly from Google Streetview. There isn’t a separate Geoguessr dataset, it just pulls from Google’s API (at least that’s what Wikipedia says).

[ Parent | on HN ]

Re:Terence Tao: Quantitative AI progress needs accurate and ... by bluefirebrand on Friday July 25, 2025 @11:38AM (#44684404)

I suspect that Geoguessr's dataset is a subset of Google Streetview, but maybe it really is just pulling everything directly

[ Parent | on HN ]

Re:Terence Tao: Quantitative AI progress needs accurate and ... by bee_rider on Friday July 25, 2025 @01:13PM (#44685567)

My guess would be that they pull directly from street-view, maybe with some extra filtering for interesting locations.

Why bother to create a copy, if it can be avoided, right?

[ Parent | on HN ]

Re:Terence Tao: Quantitative AI progress needs accurate and ... by clbrmbr on Friday July 25, 2025 @10:11PM (#44690705)

Yet.

This is a good rebuttal when someone quips that we “are about to run out of data”. There’s oh so much more, just not in the form of books and blogs.

[ Parent | on HN ]