News for hackers. Stuff that matters.
Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by orbital-decay on Tuesday September 15, 2026 @06:15PM (#49719601)
Yeah but what stops it from producing confidently incorrect outputs...
Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by zenlikethat on Tuesday September 15, 2026 @07:16PM (#49720154)
Nothing, but imagine using LLMs for a classification task

People out there are so resigned to the models being unreliable that they are really doing things like hallucinating deliberately, and then matching the hallucinations to embeddings -

https://softwaredoug.com/blog/2026/08/10/hypothetical-classi...

You could do that or you could just... use a model that will never produce unreliable outputs in the first place.

Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by threecheese on Tuesday September 15, 2026 @07:31PM (#49720272)
But we're going from "Apple" to "Apple: 99% - trust me". It could still be an image of an orange :)
Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by zenlikethat on Tuesday September 15, 2026 @07:36PM (#49720316)
It's pretty darn smart. If you did want to hack on it in earnest and find out for yourself, send me an email - [email protected]
Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by threecheese on Thursday September 17, 2026 @11:14AM (#49742033)
Oh I'm on the waiting list, I was just being a pedantic prick. I can't wait to try it.
Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by nkozyra on Tuesday September 15, 2026 @11:14PM (#49721702)
I'm certainly not resigned to that, at least for classification.

Even non-frontier models are absurdly good at this in a broad sense.

Which would make it hard to judge "a model that will never produce unreliable outputs in the first place" against something that is already really, really good and exceptional in domain-specific areas with the tiniest amount of elbow grease.

Speed and cost look good though (for now)!

Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by ActivePattern on Thursday September 17, 2026 @11:41AM (#49742459)
It would be great to see benchmarks for Jev that demonstrate the value of calibrated uncertainty.

For example, one could set a confidence threshold over which we trust the model decision, and otherwise reject. This provides a lever to trade-off accuracy and automation %.

Then we can ask questions like "What % of decisions can we automate to achieve 90% accuracy"?

Re:Jev: New frontier model 40-400x cheaper and 20-200x faster by suraj_phanindra on Monday September 21, 2026 @07:22PM (#49794787)
i ran some interesting experiments on jev today and wanted to share the results. i compared cost and latency of support ticket triage with different arrival rates for jev vs mainstream LLMs: https://suraj-website-eta.vercel.app/blog/what-a-correct-dec...