Right; just use git so you have backups, and be aware of access controls and sandboxing. But do you expect a law prof to know how to use git, have a mental model of LLM agent access policies, etc?

Um, yeah? If the professor bought a chainsaw or a bulldozer I'd expect them to read the manual on that, too.

Your Smart Bulldozer Pro is AI and can make mistakes.

There have been enough of these that anyone who reads tech news ought to know not to turn an LLM tool loose on the only copy of important data, and to take frequent snapshots when using them in larger projects. The worst I've had one do is git revert its current session.

The victim in this case was a law professor, who might not be in the habit of reading tech news.

It ought to be obvious to anyone who was born more than 2 days ago. You don't need to be an AI guru to understand that AI can do stupid things anymore than you need to be a lumberjack to appreciate the potential risks of chainsaws.

Law professors may know little of tech but they should probably know better than anyone the dangers of leaving written instructions open to interpretation without well defined limits.

Except that these are foreseeable consequences if you put tools with sharp edges in the hands of laypeople. It would be acceptable if they were advertised as such and training in their use recommended or required beforehand. But this is not how Anthropic and OpenAI advertise their tools; in fact, Anthropic touts and recommends its auto-review mode. I see analogies (admittedly, imperfect ones) to the behavior of tobacco companies here.

Insofar, despite the article being somewhat simplistic if you have a background in IT, I find it important that the risks of these tools are shown to the public if Anthropic and OpenAI won't do that themselves.