Re:In 27,280 coding agent runs, 64% of inference spend went ...
by jrmg on Tuesday November 17, 2026 @11:26AM
(#50533789)
What does "last-mile fine-tuning on [a] harness" mean?
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by cronin101 on Tuesday November 17, 2026 @11:29AM
(#50533829)
Taking the open weight model and altering those weights in order to overfit for success on the tool shape / quirks of the harness you are running the model within.
Last-mile aka doing in yourself (on box) vs the compute cluster that created those open weights.
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by ford on Tuesday November 17, 2026 @11:32AM
(#50533887)
I don't think this is common, though it would be interesting if ThinkingMachines/HuggingFace/OpenRouter/BaseTen offered fine-tuned variants of the most popular open models that worked with specific harnesses
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by cronin101 on Tuesday November 17, 2026 @11:37AM
(#50533989)
In hindsight I think I overestimated how common/achievable this is with the logic that if PewDiePie could do it, so could a layman https://felloai.com/pewdiepies-ai-model-beats-gpt-4o/
However, the hardware requirements are still quite enormous.
I could see a gap in the market for crowdfunded (gofundme?) post-training of open-weight models, per harness. I estimate that the one-time compute cost is probably $x000 so this scales nicely with a large number of interested developers that could reuse the weights.
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by hedgehog on Tuesday November 17, 2026 @12:07PM
(#50534539)
It's typically easier to tune the harness around the models' existing behavior. In my experience the primary failures in models > 5B params on coding tasks are just that they aren't able to translate instructions into a correct description or implementation of the problem (will write wrong code, wrong tests, and wrong documentation). The basic tool calling and syntax of generated code is fine.
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by bisonbear on Tuesday November 17, 2026 @12:16PM
(#50534707)
In MiMo's v2.6 public RL training run, they explicitly trained on 23 different harnesses in order to minimize overfitting on any single harness. I imagine that this technique has to be common in RL training runs given that model providers have little control over where their models are used.
Can anyone confirm/deny this is the case?
If so, it seems a shame not to include this practice into the reporting methodology for full transparency.