News for hackers. Stuff that matters.

A comment on AccountingBench: Evaluating LLMs on real long-horizon business tasks. Parent comment

Re:AccountingBench: Evaluating LLMs on real long-horizon bus... by lucianbr on Monday July 21, 2025 @03:25PM (#44639331)
You're saying P=NP, I think.
[ Parent | on HN ]
Re:AccountingBench: Evaluating LLMs on real long-horizon bus... by asgraham on Tuesday July 22, 2025 @02:58AM (#44643960)
Not to agree with GP, but I think it’s more accurate to say they’re saying “if validation is quick (to code), who cares how long a solution takes an AI because computation is cheap.”

They’re not really making any claims about how quickly the AI can solve relative to the validation, which is what P vs NP is about.

[ Parent | on HN ]
Re:AccountingBench: Evaluating LLMs on real long-horizon bus... by jjmarr on Tuesday July 22, 2025 @05:12PM (#44653073)
This.

Yes, Cline generates a bunch of junk very quickly and makes dumb mistakes. If I can filter out those dumb mistakes automatically, I don't have to supervise it. It might burn 20¢ on fixing a mistake no human developer would make, but I don't have to pay a human developer.

[ Parent | on HN ]