you can in principle make a hook in the harness itself that will run an auditor prompt that checks it adhered to the policy, correct the model and also make it known in advance.
https://github.com/anthropics/claude-code/blob/7ef6eec9d9ba8...
That's why it's critical to have these checks run in a context that the model can't bypass, such as via github actions that block PR merges. Generally works pretty well for stuff at work (where we have all this stuff set up as part of continuous integration checks), but it means that for personal stuff I have to set up quite a bit more infrastructure to ensure that models don't just skip running tests (which they LOVE to do).
These seem like they would naturally extend to LLMs as well.
(I'm not pro-LLM on balance, but the solutions of today feel similar for humans and LLMs.)
In this case, it is an advisory control only, because the LLM can also unhook that hook. And of course, it could also just disable the failing test(s) with some bullshit reason. But it is far better than assuming it will comply every time.
Then there is the "hard control", which is the inviolable that the LLM cannot bypass.
You need to move as much as is technically possible to either hard or (failing that) advisory controls.