One thing I noticed when I first started with the harness and local LLM was that how I was writing the initial specification needed to change. By watching the thinking of the LLM I gradually honed the specification of what I wanted until the point where it was able to one shot it correctly. Then that spec fed to deepseek, GLM, smaller LLMs even like Ornith 1.5 they all managed to one shot it but with clear differences in the quality of what they did.

I definitely found value in this approach by seeing what the LLM right and wrong was a large part about what the specification said. Fixing those issues also reduced the LLM thinking tokens. The specification was better. This is a similar style of process to what is used here, but really learning to adjust to what the LLMs do well and don't do well from interpreting the input seemed to be key.