It's fucking depressing. I have a skill that says it's ok to connect to a test server on VPN that gets wiped out each midnight with user/password and it works on Monday but is blocked on Tuesday with some idiotic excuse like credentials exploration or whatever.

And then I tell it to connect to a SSH server it knows nothing about and it's like sure, why not?

> but their preflight safety classifiers are a dumpster fire

Even more annoying: They can be easy to subvert.

I was running something at work that involved an LLM, and kept getting safety refusals. The weird thing was that when I went to our in-house chatbot, set the model to the same one the program was using, entered the same text, it happily answered.

I spent quite a while trying to figure out why my program was being refused. Was I doing something wrong?

No - the chatbot was doing something "right". You see, the chatbot had some tools it injected into the prompt (current time, etc).

So I went back to my program and injected the same tools. Suddenly my request was very safe.

The phrase "security theater" was invented for stuff like this.

I switched to DeepSeek flash v 4 something and the difference could not be starker. And I never seem to run out of "tokens" even at the highest reasoning level on all the time