News for hackers. Stuff that matters.
Re:Jev Does Not Play Dice: 83% probability, 19% accuracy on ... by kantahayashi on Thursday September 24, 2026 @09:56AM (#49830671)
I used a die as an example of problems whose answers can't be known at all, unlike problems with correct answers like MMLU questions. In a more practical test, a document said "30% risk" and Jev returned 5%. The point of the post is that Jev is weak at some kinds of tasks, so you should check the calibration for your own use case.

There are no comments on this comment yet.