Re:In 27,280 coding agent runs, 64% of inference spend went ...
by Edward40 on Tuesday November 17, 2026 @12:03PM
(#50534473)
We want to see how performance diverges between TB2.1 and TB4 while keeping costs manageable, since running the entire dataset is really expensive.
[ Parent
| on HN ]
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by seizethecheese on Tuesday November 17, 2026 @12:07PM
(#50534538)
How were the specific questions chosen?
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by Edward40 on Tuesday November 17, 2026 @01:07PM
(#50535581)
You can see the task in https://hub.harborframework.com/datasets/frontier-harness-ev...
We kept the category unified so we have tasks for every domain like ML , science and others.
Re:In 27,280 coding agent runs, 64% of inference spend went ...
by seizethecheese on Tuesday November 17, 2026 @01:44PM
(#50536192)
[delayed]