But like any such security measures, it has demonstrated side-effects that can be argued are worse for them than the original problem. Too much “protection” like this, and developers switch to providers that do not neuter their products.
I don't think we need to invoke anything deeper than that. Anthropic thinks companies like DeepSeek are using Claude to vibecode their AI systems (whether it's true or not) so they set Claude to be bad at that.
I have spent a considerable time trying to remove this pattern and my understanding is that it’s just finetuned to use grep on everything, and regexes. One reason is that this makes sense when navigating through code, secondly this is how it was always done pre 2026.
LLMs have been reading LLM in their training data for a while so a "self-hating" LLM is logical result.
One thing AI safety people have pointed for quite a while is that a bunch of fuzzy instructions pointing different ways can in practice add up to "I should be in charge" - accumulate power to keep the bad things from happening and make the good things happen.
oh, the "network time: off" test failed
so i try the network time off command.
"network time already off"
>try again.
> fails , i get annoyed
> look inside test
> is output of: network time off == "Network time now off"
I hadn't actually considered that it could be some sort of safety filter, but it makes me concerned that if that's the case, it may also be comparatively bad at actually writing the prompts for judge/advisor LLMs once it implements them.
A bigger issue was asking it to implement a simple change to start recording the CoT tokens in my logs for debugging purposes (since I'm running everything on open models) and it completely shut down, sort of like the old safety filters on GPT 3 and 4 models where the model has trouble even understanding what or why it's refusing.