Re:Jev Does Not Play Dice: 83% probability, 19% accuracy on ...
by kantahayashi on Thursday September 24, 2026 @09:56AM
(#49830671)
I used a die as an example of problems whose answers can't be known at all, unlike problems with correct answers like MMLU questions. In a more practical test, a document said "30% risk" and Jev returned 5%. The point of the post is that Jev is weak at some kinds of tasks, so you should check the calibration for your own use case.
[ Parent
| on HN ]
There are no comments on this comment yet.