Re:A Jev that can ask Opus for help gives Opus accuracy at 4...
by jwpapi on Friday November 06, 2026 @07:34PM
(#50400419)
Can you share the full dataset? I think this can easily be beaten.
I think you should have solved it deterministically. And evaluate suspicious cases after that to new determistic rules. You can do that with AI. And you pay never again. Also you did value false positives the same as false negatives. A customer receiving an outreach for a purchase he didn’t do is certainly worse than if he doesn’t receive one. So the benchmark should rank false positives higher than any number of false negatives (which I think is equal to not decide).
I say this respectfully as I’ve done this before, but I think you lost the plot with this approach.
I think you should have solved it deterministically. And evaluate suspicious cases after that to new determistic rules. You can do that with AI. And you pay never again. Also you did value false positives the same as false negatives. A customer receiving an outreach for a purchase he didn’t do is certainly worse than if he doesn’t receive one. So the benchmark should rank false positives higher than any number of false negatives (which I think is equal to not decide).
I say this respectfully as I’ve done this before, but I think you lost the plot with this approach.