Re:AMD acquires Taalas to boost inference performance by etc...
by wmf on Thursday August 06, 2026 @07:25PM
(#49204011)
There's a benchmark for this and a lot of models get negative scores because they're so unreliable: https://artificialanalysis.ai/evaluations/omniscience
[ Parent
| on HN ]
Re:AMD acquires Taalas to boost inference performance by etc...
by daishi55 on Thursday August 06, 2026 @09:26PM
(#49204848)
I wanted some examples they actually experienced. Because I use these things daily and haven’t seen a hallucination in a long long time.
Re:AMD acquires Taalas to boost inference performance by etc...
by tyre on Thursday August 06, 2026 @11:59PM
(#49205757)
Search a terminal with Claude Code for things like, “I got it wrong twice. I should look up the documentation instead of guessing.”
Does it about once a day, that I notice.
Re:AMD acquires Taalas to boost inference performance by etc...
by yoyohello13 on Thursday August 06, 2026 @09:42PM
(#49204949)
I saw a hallucination just this afternoon about a spurious ca cert error. Definitely happens less often, but I do need to correct it occasionally. Maybe once a week so it still requires vigilance.
Re:AMD acquires Taalas to boost inference performance by etc...
by B-Con on Saturday August 08, 2026 @03:03AM
(#49219504)
Not OP but I've tried using all the major frontier models to find niche items in a field that I collect. The models get confused and hallucinate items that sounds complete reasonable, but don't actually exist. Usually it's an amalgamation of several real products.
My prompt is akin to "recommend <item type> with <niche criteria>". The first 3-ish results are about right, and then 7 of the next 10 are hallucinations and the LLM clearly can't throw up its hands and say "I got nothing".
I'm sure this is a hard problem because of a) how many items there are, b) how much overlap there is between product names, descriptions, manufacturers, different versions of the same product, etc, so keeping them distinct in the model's memory is probably hard, and even worse if it is dynamically fetching and summarizing content then it will be very easy to conflate different items, and c) LLMs are known for not working well on the edge cases with few examples.