News for hackers. Stuff that matters.
Re:Show HN: Spegel, a Terminal Browser That Uses LLMs to Rew... by mossTechnician on Tuesday July 01, 2025 @11:53AM (#44435170)
Changes the AI browser made to the ingredient list:

Pounds of lamb become kilograms (more than doubling the quantity of meat), a medium onion turns large, one celery stalk becomes two, six cloves of garlic turn into four, tomato paste vanishes, we lose nearly half a cup of wine, beef stock gets an extra ¾ cup, rosemary is replaced with oregano.

Re:Show HN: Spegel, a Terminal Browser That Uses LLMs to Rew... by achierius on Tuesday July 01, 2025 @11:54AM (#44435185)
Did you actually observe this, or is just meant to be illustrative of what could happen?
Re:Show HN: Spegel, a Terminal Browser That Uses LLMs to Rew... by mossTechnician on Tuesday July 01, 2025 @11:58AM (#44435223)
This is what actually happened in the linked article. The recipe is around the text that says

> Sometimes you don't want to read through someone's life story just to get to a recipe... That said, this is a great recipe

I compared the list of ingredients to the screenshot, did a couple unit conversions, and these are the discrepancies I saw.

Re:Show HN: Spegel, a Terminal Browser That Uses LLMs to Rew... by orliesaurus on Tuesday July 01, 2025 @11:59AM (#44435234)
oh damn...
Re:Show HN: Spegel, a Terminal Browser That Uses LLMs to Rew... by jugglinmike on Tuesday July 01, 2025 @12:20PM (#44435464)
Great catch. I was getting ready to mention the theoretical risk of asking an LLM be your arbiter of truth; it didn't even occur to me to check the chosen example for correctness. In a way, this blog post is a useful illustration not just of the hazards of LLMs, but also of our collective tendency to eschew verity for novelty.
Re:Show HN: Spegel, a Terminal Browser That Uses LLMs to Rew... by andrepd on Tuesday July 01, 2025 @12:57PM (#44435877)
> Great catch. I was getting ready to mention the theoretical risk of asking an LLM be your arbiter of truth; it didn't even occur to me to check the chosen example for correctness.

It's beyond parody at this point. Shit just doesn't work, but this fundamental flaw of LLMs is just waved away or simply not acknowledged at all!

You have an algorithm that rewrites textA to textB (so nice), where textB potentially has no relation to textB (oh no). Were it anything else this would mean "you don't have an algorithm to rewrite textA to textB", but for gen ai? Apparently this is not a fatal flaw, it's not even a flaw at all!

I should also note that there is no indication that this fundamental flaw can be corrected.

Re:Show HN: Spegel, a Terminal Browser That Uses LLMs to Rew... by throwawayoldie on Wednesday July 02, 2025 @09:54AM (#44443768)
> the theoretical risk of asking an LLM be your arbiter of truth

"Theoretical"? I think you misspelled "ubiquitous".

Re:Show HN: Spegel, a Terminal Browser That Uses LLMs to Rew... by simedw on Tuesday July 01, 2025 @12:54PM (#44435844)
Fantastic catch! It led me down a rabbit hole, and I finally found the root cause.

The recipe site was so long that it got truncated before being sent to the LLM. Then, based on the first 8000 characters, Gemini hallucinated the rest of the recipe, it was definitely in its training set.

I have fixed it and pushed a new version of the project. Thanks again, it really highlights how we can never fully trust models.