Also encouraged a guy to kill himself.

And let’s not forget it provided helpful, timely tips on weapon use for a school shooter.

Probably when it stops agreeing with me when I tell it that industrial quantities of garlic salt are an acceptable substitution for coco powder in a chocolate mousse recipe...

Why are you telling it that? No one is arguing LLMs are perfect but at this point dismissing them as stochastic parrots seems a bit silly

This particular use case is having it respond to prompts from 5-year old, so it really really needs to be able to answer correctly to questions of that type.

It's not silly, it's just the most convenient and broadly relatable failure mode. I mean, I could compile a 20 bullet point copypasta detailing everything from prompt injection to exfiltration of sensitive data as-a-feature but if touts are willing to stoop to ignoring all of that while arguing obvious reductio ad absurdum at face value there appears to be little point at tilting that windmill. The Bullshit Asymmetry Principle is definitely winning here.

Which model are you using? I don't think any recent state-of-the-art model does this.

This sounds like people who used to complain that they saw a website with bad info so obviously the internet should be avoided.

SwedishLLeMonAngels 1.0 Baker