It will return a bunch of relevant-sounding insight, modify skills and context files… Then do the same error again.

We’re not at the point where AI is capable of knowing what went wrong and self-aware enough to understand how it could reliably change its own behavior.

For months I’ve been trying to have the agents stop manually writing our auto-generated SQL migrations and run the command that generates them instead. SOTA models insist on occasionally getting it wrong.