A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost

Re:A Jev that can ask Opus for help gives Opus accuracy at 4... by nico on Friday November 06, 2026 @04:47PM (#50398951)

This is very cool. But also, it’s a benchmark/result for a very specific experiment/task: identity resolution

Despite the title, the article is not making claims of, or demonstrating general usage improvements or general coding agent improvements

[ Parent | on HN ]

Re:A Jev that can ask Opus for help gives Opus accuracy at 4... by theanomaly on Friday November 06, 2026 @05:05PM (#50399123)

Agreed, though many kinds of tasks are likely to benefit from this pattern, YMMV.

[ Parent | on HN ]

Re:A Jev that can ask Opus for help gives Opus accuracy at 4... by jwpapi on Friday November 06, 2026 @07:34PM (#50400419)

Can you share the full dataset? I think this can easily be beaten.

I think you should have solved it deterministically. And evaluate suspicious cases after that to new determistic rules. You can do that with AI. And you pay never again. Also you did value false positives the same as false negatives. A customer receiving an outreach for a purchase he didn’t do is certainly worse than if he doesn’t receive one. So the benchmark should rank false positives higher than any number of false negatives (which I think is equal to not decide).

I say this respectfully as I’ve done this before, but I think you lost the plot with this approach.

[ Parent | on HN ]

Re:A Jev that can ask Opus for help gives Opus accuracy at 4... by pkulak on Friday November 06, 2026 @10:27PM (#50401352)

Not hard to test. I wired a “judge” into my existing Jev setup and… it works. I was able to trim my prompt down to the basics without “engineering” and have my entire test suite pass. Best I could get before was 1 failure, with an extremely brittle prompt. Like, adding a comma where grammatically appropriate broke 10 more tests.

[ Parent | on HN ]