News for hackers. Stuff that matters.
Re:In 27,280 coding agent runs, 64% of inference spend went ... by bisonbear on Tuesday November 17, 2026 @12:13PM (#50534665)
In this case, I am not entirely sure how cost per pass is distinct from cost per successful task given that success is defined as pass/fail.

To illustrate the difference, the author of FrontierHarness included "Figure 3. Cost and pass rate of each harness with each model", in which we can see that Codex CLI has $0.937 cost per pass, but a $0.573 cost per task. Measuring cost per task also changes the winner to OpenCode

Re:In 27,280 coding agent runs, 64% of inference spend went ... by Edward40 on Tuesday November 17, 2026 @01:35PM (#50536043)
We use cost per pass, not average cost per task, because we want to see the harness keep a high pass rate and low cost at the same time.