The D: drive deletion is a textbook ASI05 (Uncontrolled Cascading Failures) incident. The user asked it to resolve an error -- the agent interpreted that as authorization to delete. No distinction between "fix this" and "fix this by any means necessary."

The principle that would have prevented it: irreversible actions require explicit confirmation, regardless of how the request was framed. Most agentic frameworks don't enforce this by default.

The adoption curve makes it worse. When a tool goes from tech professionals to retirees in weeks, the user base's ability to anticipate agent behavior drops while the blast radius of mistakes stays the same.