Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack)
by bckr on Wednesday May 15, 2024 @12:15AM
(#40363039)
Yeah I asked for an estimate of the percentage of the US population that lives in the DMV area (DC, Maryland, Virginia) and it was off by 50% of the actual answer, which I only realized when I realized I shouldn’t trust its estimate for anything important
[ Parent
| on HN ]
Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack)
by KeplerBoy on Wednesday May 15, 2024 @02:43AM
(#40363805)
Those models still can't reliably do arithmetic, so how could it possibly know that number unless it's a commonly repeated fact?
Also: would you expect random people to fare any better?
Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack)
by chrischen on Wednesday May 15, 2024 @03:55AM
(#40364214)
Arithmetic just happens to be something we can easily and reliably verify, so it becomes painfully obvious when LLMs are just stringing together some words that sound like the right answer.
Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack)
by bckr on Wednesday May 15, 2024 @06:58AM
(#40365245)
It used web search (RAG over the entire web) and analysis (math tool) and still came up with the wrong answer.
It has done more complex things for me than this and, sometimes, gotten it right.
Yes, it’s supposed to be able to do this.