Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview

Re:Show HN: OSS Agent I built topped the TerminalBench on Ge... by Mashimo on Monday April 27, 2026 @09:18AM (#47921202)

Interesting. Would love a comparison to pi.dev (Not Ohmypi)

How does this perform in day to day coding tasks, outside of benchmarks?

[ Parent | on HN ]

Re:Show HN: OSS Agent I built topped the TerminalBench on Ge... by GodelNumbering on Monday April 27, 2026 @09:24AM (#47921258)

https://github.com/dirac-run/dirac#-evals

README has eval of 8 tasks over 7 agents (including both pi and omp). Pi-mono costs second lowest across the 8 tasks (after Dirac) but occasionally misses produces incomplete changes.

Interestingly, 2 tasks where pi missed some changes both were the tasks that benefitted from AST symbol understanding (e.g. find all instances of things that refer to this symbol and change those things). Since pi relies on bash type tooling, it missed some occurrences

[ Parent | on HN ]

Re:Show HN: OSS Agent I built topped the TerminalBench on Ge... by howdareme on Monday April 27, 2026 @09:48AM (#47921521)

Going to assume you didnt capture the data but could you add time taken to completion for each if you have it?

[ Parent | on HN ]

Re:Show HN: OSS Agent I built topped the TerminalBench on Ge... by messh on Monday April 27, 2026 @10:47AM (#47922337)

re. bash type tooling-- it doesnt mean an agent cannot use ast: using treesitter cli this should be perfect possible

[ Parent | on HN ]

Re:Show HN: OSS Agent I built topped the TerminalBench on Ge... by Cilvic on Monday April 27, 2026 @04:49PM (#47927125)

I assume that this benchmarks where done without any modifications to the default open-sourced harness. treesitter CLI would be an extra plugin for pi-mono, put I'd be equally curious about whether it would accomplish the task.

[ Parent | on HN ]