Re:Jev Does Not Play Dice: 83% probability, 19% accuracy on ...
by __m on Thursday September 24, 2026 @09:42AM
(#49830458)
why would you use ai for that though? how does that translate to other tasks?
[ Parent
| on HN ]
Re:Jev Does Not Play Dice: 83% probability, 19% accuracy on ...
by kantahayashi on Thursday September 24, 2026 @09:56AM
(#49830671)
I used a die as an example of problems whose answers can't be known at all, unlike problems with correct answers like MMLU questions. In a more practical test, a document said "30% risk" and Jev returned 5%. The point of the post is that Jev is weak at some kinds of tasks, so you should check the calibration for your own use case.