Re:Decision models like Jev don't beat LLM-as-a-judge or tra...
by wat10000 on Monday October 05, 2026 @12:56PM
(#49967378)
Probably not. But it does seem to have some idea of probabilities if I get a little more specific. "I've rolled a fair die. I see several dots on the top." Gives me 46% for 6, and about 16% for 4 and 5. If I say "The top is mostly white." then it says 1 with 50% confidence... and 6 with 31%.
Interestingly, if I add a "unknown" option then my original question says unknown with 100% confidence. "I see several dots on top" gives 89% unknown. "The top is mostly white" gives 30% unknown and 37% 1.
So it definitely has some ability in this area, but it's iffy. This isn't exactly what it's made for but it doesn't seem too far off from the examples like taking in a customer email and deciding which team it belongs with.
Interestingly, if I add a "unknown" option then my original question says unknown with 100% confidence. "I see several dots on top" gives 89% unknown. "The top is mostly white" gives 30% unknown and 37% 1.
So it definitely has some ability in this area, but it's iffy. This isn't exactly what it's made for but it doesn't seem too far off from the examples like taking in a customer email and deciding which team it belongs with.