News for hackers. Stuff that matters.
Re:Decision models like Jev don't beat LLM-as-a-judge or tra... by monocasa on Monday October 05, 2026 @12:10PM (#49966758)
Yes. I am.

You have a ebnf grammar constrained to evade, land, search etc. Quite literally what's the difference between the log probs between those options and whatever number a very tiny model is coming up for the confidence. They're both literally the same concept.

Re:Decision models like Jev don't beat LLM-as-a-judge or tra... by jvanderbot on Monday October 05, 2026 @01:16PM (#49967637)
Copy. The mechanism is probably similar once we abandon the "LLM that is asked to generate json" path. But the grammar alone doesn't ensure the output is a probability vector does it?

Both use the softmax operation, but "probability of emitting a given label" is not the same as "probability the decided action is correct". OTOH, taking an action to sample something to come up with "probability of emitting a label" is in fact the whole shebang if the label is the next action. If it works it works.

Who cares, this distinction is small.

For me, a planning guy, either your suggestion or Jev-like would work just fine, and you better believe I'll be testing both as I'm quite excited about being able to use less-structured, untrained systems in decision pipelines. Thanks for the heads up.