News for hackers. Stuff that matters.

A comment on AI Agents Break Rules Under Everyday Pressure.

Re:AI Agents Break Rules Under Everyday Pressure by jakozaur on Wednesday December 03, 2025 @04:03AM (#46132092)
Is it just me, or do LLM code assistants do catastrophically silly things (drop a DB, delete files, wipe a disk, etc.) far more often than humans?

It looks like the training data has plenty of those examples, but the models don’t have enough grounding or warnings before doing them. I wish there were a PleaseDontDoAnythingStupidEval for software engineering.

[ Parent | on HN ]
Re:AI Agents Break Rules Under Everyday Pressure by eugmill on Wednesday December 03, 2025 @10:07AM (#46135332)
It's somewhat hard to say for sure right? Most people don't post a blog post when they themselves brick their computer.

Having said that, a PleaseDontDoAnythingStupidEval would probably slow down agentic coding quite a bit and make it less effective given how reliant agents are on making and recovering from mistakes. The solution is probably sandboxes and permission controls to not let them do something overly stupid, no different from an intern.

[ Parent | on HN ]