Re:Jev in 25 Lines of Python
by nostrebored on Wednesday September 23, 2026 @02:08PM
(#49820157)
Yes, for vision classifiers we have in prod, I've seen a huge difference between A, B, C, 1, 2, 3 style answers and emitting a string. Even from just base model behavior pre-sft/rl. It was one of those obvious in retrospect moments.
The whole point of my argument is that neither is good, but from a technical perspective logprobs is probably the worst unless you train a model on specific outputs. In which case you'd throw out the generality again, so when I think about it more, it's actually the worst overall. In my experiments, having the model simply assign "high" or "low" probability in a structured output generally performs best. You can try numbers, but you will never get anything close to what you could expect from traditional ML. And most certainly not from logprobs.