Re:Jev: New frontier model 40-400x cheaper and 20-200x faster
by monkpit on Wednesday September 16, 2026 @12:49AM
(#49722213)
To be fair - you’re crafting a deliberately bad model input for a contrived example.
Re:Jev: New frontier model 40-400x cheaper and 20-200x faster
by djray on Wednesday September 16, 2026 @10:47AM
(#49727879)
No, the model has answered correctly. Your question is poorly phrased (possibly deliberately).
Your question would correctly classify the user's input as requesting a human support agent, but at an indeterminate time.
If you wanted to determine whether the user wants to speak to a human support agent immediately, you would have to correctly qualify your question, e.g. "Does the user want to speak to a human support agent now?". You could have another question which is "Is the user requesting a call-back from a human support agent?". Or you could have a multiple choice query which would filter the conversation into one of a number of pre-written possibilities.
This is nothing to do with accuracy or hallucination. It's a different method of interacting with the model where you are relied upon to be precise.
Re:Jev: New frontier model 40-400x cheaper and 20-200x faster
by matja on Friday September 18, 2026 @08:05PM
(#49761902)
Add:
"call_back_day": {
"criteria": {
"none": "The user does not want a call back.",
"today": "The user wants a call back today.",
"tomorrow": "The user wants a call back tomorrow."
},
"instructions": "When does the user want a call back?",
"type": "choice"
}
And you get: "tomorrow": confidence 1
Model input: "Does the user want to speak to a human support agent?"
Output: Yes.
I imagine that your model would produce this, and I think it's fair to say this is a hallucination. A human would caveat it with: "Yes, but not right now.", your model is incapable of that. Yes is technically correct, but within the context of being in a live chat, a human would understand that the caveat is required.