Here is something I found this week: copies the entire db to memory, inserts a single row into in-memory db, deletes everything from real db, writes everything back. Sentence that caused this is "do XXX in one transaction".

All tests pass, everything seemed to work. Would not have noticed of I didn't read through the code. Amount of other stupid shit I found is crazy.

These anecdotes mean nothing without knowing context, model, harness, prompt. The technology has moved a lot in the past year.

To that end, my very first experience with OpenCode let to a `rm -rf /` within minutes, which I caught by chance watching the streaming output. Now I know what I'm doing, every agent is sandboxed, better tooling and models.

Ah yes, the favorite response to any criticism of LLMs – you are using the wrong model.

A performance benchmark would have easily caught this though, ask the LLM to create and run benchmarks and see how those change after each new feature/major change.

I have found test coverage reports to be really helpful as well.