It's because everyone wants their 15 minutes of fame so every single time something shiny gets out all the NPCs are asking astra/fable: "can you create a copy of this, make no mistake"

Thanks to these projects, what was an innovative-but-closed piece of technology one week ago is now much more accessible. Whether they're in it for fame or not, I couldn't care less!

There already was an alternative a year ago, with a published paper and open weight lmao... all the other projects are literal slop shat out by script kiddies 2 hours after the release of jev, it reminds me of the flappy bird era, depressing

It’s because it’s practically useful and enabled things that were impractical previously.

> and enabled things that were impractical previously

I think that there are not _that_ many use-cases that have been opened up by this that tool-calling on other models didn't solve already. Really depends what benchmark you're looking at. This one against BANKING77[0] has many issues, but suggests it's really not far off DeepSeek 4.1 Flash. This one against BoolQ[1] shows marginal improvement over Qwen3.6. This one against MMLU-Pro[2] (same author as the previous) shows significant improvements over two Qwen models.

So there's definitely _some_ alpha there, but I don't think it's the sea-change that the hype would suggest; that is to say, yes, some things that weren't practical before are now, but many things were already very practical with the existing tools.

0: https://sanand0.github.io/llmevals/jev/

1: https://github.com/ekzhang/openjev-sglang/blob/a3554ed9e9c26...

2: https://github.com/ekzhang/openjev-sglang/blob/a3554ed9e9c26...

The part about "tool-calling on other models didn't solve already" is what gets you, sure I could tool call deepseek, glm or any other model, but the latency is huge and you get no confidence score. I gave JEV a shot via OpenRouter and it has a reply in less than 400ms, it's fast enough and cheap enough that you can hook it up to a game loop for example (so highly state dependant) and it can do decisions in real time.

Please refer to the latency and price figures for DeepSeek V4.1 Flash in the first link I shared.

It's appealing not having to fine-tune separate model for each use case

So you have more flexibility to get on with building, evolve your business logic etc

It reminds me a bit of what Ansible got right: user communication. The underlying tech may have existed for a long time, but the genius is presenting it to a regular developer in a way that reads "yes, even you can understand ML, just using a little JSON". The contribution of that should not be understated, as has been clearly evident recently.

Yeah except it doesn't really work. It constantly breaks underneath you. The whole system has to be managed, e.g NixOS, or else it's a house of cards.

I'm also not personally a fan of Ansible, but to claim it doesn't really work is quite breathtaking given the size of the installed base.

Arguably, it doesn't. Instead of a team of people doing break-fix on golden images you have that same size team of people doing break-fix on upstream playbooks. Lots of software doesn't achieve the goals but has a huge deployment story; not sure those things have ever been related.

> Yeah except it doesn't really work

It does "work", you can download ansible today and use it, it does what it says. Is it the greatest solution for all use cases in infrastructure? Of course not, nothing is. Do people misuse it? Of course too, we're all human.

Regardless of what tooling you use, we're all building houses of cards, and depending on the situation, try to hold down those cards as well as we can, balancing a ton of other needs and requirements.

The way I see this (I havent played around with Jev or layla the OSS version) is that classifiers have always existed and a recognised tool in the ML world. But, the norm is that one needs to not only know what to classify as, but determine what weights to use to classify the input.

Jev came in, and added that magic of "you dont need to train your classifier or determine the weights" if you dont want to, and just get the classified answer out. I think that's what is making people see this with a glitter in their eyes.

I would be curious to see comparisons of jev and similar things with problem specific classifiers. I think layla suggested making problem specific versions anyway? There is a lot of demand for magic don't do any work solutions, which is kind of weird in an era where agents can really help you build a customised solution effectively.

Yes, I've been enjoying playing with Jev. But I do keep wondering: maybe I could just train my own classifier for each of these things I'm doing?

But it's still pretty nice to have an extremely low barrier to entry to prototype something involving a classifier. I keep being like "hmm, that seems useless actually, let's try something else". It's nice to be able to get to that point before needing to gather all the data and create a bespoke classifier.

Agreed.

Just to be helpful if anyone is searching for layla, it's laya.

Jev is creating a sort of identity crisis for me, because the number of absolutely clueless folks parroting the classifier thing is the first time I've seen this sort of mass psychosis in CS upfront.

Like even 5 minutes of tinkering captures why this isn't anymore like BERT or any past classification model than ChatGPT is like those old Markov Chain generators, yet folks cannot shut up about how this is nothing new.

Absolutely scary and makes me wonder how much of the field is just people super confidently discrediting otherwise promising/interesting directions for development for a cheap dunk!

Hey I am clueless, how do I learn more?

Why is Jev fundamentally better than classification models like BERT or traditional ML?

Happy to read a written response or if you suggest a prompt to put into my LLM to get it to research and explain the relevant details.

You already wrote the prompt, no? What I'd do, if I were you, is run the question through a LLM and then come back with targeted questions that it didn't answer.

I did the first part yesterday, jumped down the rabbit hole, and have 3 product ideas in my head now.

"Why is Jev fundamentally better than classification models like BERT or traditional ML?"

Please see my other reply, I was not exaggerating when I said this feels like asking why ChatGPT is different than a Markov Chain.

You're going to post-train 100s of instances of BERT? Traditional ML had world knowledge more than a fart?

The closest/fairest comparison is still an LLM, but no one has actually chucked enough compute at post-training to make a better Jev yet.

I'm sure in more time that'll happen, and so my excitement is expanded to Jev-like things... but so far most Jev like things are this weirdly reactionary attempts to steal thunder: is it so bad if we have some team actually invest in a quality post-training receipe to compete?

Show me a single example how is this jev thing better than a modern Bert solution?

Or even llm if you claim about versatility. You can easily modify the llm inference code to make it predict a single token represent the classification choice and extract the probability that way.

Sure jev will still be faster, but a local deployed Bert model is way faster than both.

And to get the most out of it you still need to fine tune the models anyway, unless your classification task is just one of those mainstream ones.

I did a GRPO run (multiple now actually) with a per sample rubric that leans heavily on subjective preference judgements that BERT wouldn't have the learning capacity for: not to metion you'd need to finetune hundreds of instances and host them somewhere.

And even if BERT wasn't woefully underintelligent for the task... have 100+ instances of BERT running locally faster than Jev API response times? Sweet rig you must have...

LLMs would not be fast enough without constrained decoding tricks that people fundamentally don't seem to understand make the models much dumber, and sure wouldn't be cheaper or faster.

Again I feel this deep discomfort because presumably you're somewhat intelligent but your opening salvo made it hard not to scream DO YOU EVEN HAVE A SINGLE CLUE WHAT IT DOES instead of giving you my actual answer... yet you're speaking from the chest! If I didn't try it for myself I would have been 100% sucked into you and this ocean of clueless negativity.

-

I apologize if that sounds harsh but it angers me because why should I have to deal with this kind of noise in an already insanely noisy environment? What do you gain from being cluelessly pessimistic?

And dwelling a but more I think it breaks one of my most used filters which was assuming people who know the "old world" of AI/ML are better at judging the "new world" full of hype and noise. Maybe my frustration is also just fear that things moved so quickly that the "old world" is becoming increasingly irrelevant. That'd be really disappointing.

> And even if BERT wasn't woefully underintelligent for the task... have 100+ instances of BERT running locally faster than Jev API response times? Sweet rig you must have...

why the heck do you need 100+ instances of bert. do you even attempt to research about this before?

the laya paper show that you can do the similar stuff with jev using modern bert only: https://laya.convaiinnovations.com/

and even without the newer wave of applying llm techniques to the older bert models, even flan-t5 was trained for handling 1800+ tasks.

Finetune and infer: One instance ModernBERT didn't have the learning capacity for a single problem in the shape of my subjective preference task with finetuning, do you not have the basic research taste to realize no conceivable post-training recipe will result in an instance that can zero-shot hundred plus similar questions that vary with each sample?!

You really need to try that to find out?

And again have you actually tried Jev? It has a ton of world knowledge: it's able to infer user personas based on TV show watch histories using fairly recent titles... where the hell do you think that capability is emerging in 395M params?

The irony is if you really want to die on this hill, there are much better angles by focusing on LLMs that've had diffusion heads attached for fast inference with as much of a constrained decoding intelligence penalty: at least that'd put you in the ballpark.

I was being charitable that you know the field and are clueless about Jev, mea culpa for giving you the space to think I'm the one that's missing something.

For a while is the keyword. It’s just vibe coders have just discovered the classifiers

Feels like astroturfing.

I think it’s timing. So many devs trying to squeeze their subscriptions, build more tooling to throughputMaxx. 6 months ago, i speculate it launches pretty flat.

Classifier models are extremely niche and trained for a singular purpose. A utility classifier that you can one-shot on almost any topic or need is a dramatically different beast.

Is it truly useful or accurate or beneficial? To be seen. But it's the idea that has everyone so captivated. An expert system that is an expert at most everything is a lot more useful than an expert system that is an expert at choosing a bar of soap, for instance.

Could you link to some of these classification models that can be used as versatile and perform with similar quality, speed and cost?

The Jev model is economically, but also in terms of compute, a much more efficient model. A normal LLM goes token by token, each token in a separate step. Whereas Jev just returns all the results the first round. So it is much better at classification than LLMs.

Compared to traditional ML classification, Jev works without training, like a LLM.

It’s simply not understanding. In terms of latency, LLM processing for large fact-based decision-making was incredibly slow. The real unlock here was speed in decision-making.

We know how useful classification models are, but massive pretraining is even more useful. So useful that it's tempting to just use a generic autoregressive model for everything, rather than trying to train a specialized classifier.

I think specialized models of all sorts, if it were possible to train them on equally much data, would outperform general models. In cases where we can have the data, like in self-play in games, they clearly do and have done so for a long time.

But it's a lot of effort to train specialized models. I think it's too early to tell if Jev delivers on its promises to give the best of both worlds, but if it doesn't, we'll have to make it work. It's just the right thing to try.

Previous classification models need to be trained on the specific question/choices you are trying to output. Jev doesn't need to be retrained for every choice set provided.

LLMs can act as classifiers but they still have to generate text output in the form of a JSON object. This means they have to generate every single curly bracket, quote, command, etc. This turns out to be pretty expensive. On the other hand, Jev uses a different decision head so it doesn't generate text output at all, it outputs logits *only* for the choices provided. So it completely avoids the need to generate text at all, which means no malformed JSON and it's much faster as a result.

Finally, Jev also provides confidence scores that are actually reliable (not made up like LLMs).

>Classification models have been around for a while.

I'm still trying to catch-up on the Jev stuff, but my understanding is that it's basically just a more efficient LLM when all you want is the LLM to produce a classification.

There's more to it, of course, but it's not just "generic" classification ML because it accepts arbitrary inputs and can produce probabilities over arbitrary classes. Not saying this is the first time people have done this, but typically classification tasks are more static and limited.

In the same vein, it's also not just an LLM with structured outputs (which have been a thing for a while) specifically because that is a very inefficient way to approach classification using this kind of architecture. Jev models are much more performant because of how limited they are compared to a full LLM.

So when you want an LLM, but you only really need this kind of classification from the LLM, then Jev makes a ton of sense. This makes sense for me, since I've definitely used LLMs for this kind of classification work and, even then, it kind of felt like using a jackhammer to place some nails, etc.

Happy to be correct, though.

> Happy to be correct, though.

Not normally one to point out a typo but this one made me smile

But an LLM provider could very easily add a "Jev mode" to any existing model, right? LLMs already produce a probability distribution over arbitrary classes. Just tell e.g. 5.6 Luna “here is the user's question, you must respond ONLY with the words 'foo', 'bar', or 'baz',” run a single forward pass of the model, and report the normalized probabilities of 'foo' 'bar' and 'baz' tokens before the first output.

With such an approach you could even retain full reasoning capability

My hunch is that you would need some post training. On top of that; I don’t think the llm itself can read inside the transformer state although I can see how that could be enabled. (I feel that would open up yet another class of exfiltration opportunities)

If you did you still wouldn't match Jev on price and speed though. it's orders of magnitude

I think the reason is in the general ballpark of people throwing LLMs at a huge variety of problems and being too slow disqualifies them from a bunch of things.

Now there's a new training-free thing that is fast enough to be useful on a new class of problem.

If you have a little data and can ask a frontier LLM to train a model, you can probably beat it on average for a specific task.

But... This is the case with LLMs too.

I do think "I already have an intuition for how to use and compose this!" has been an exciting thing for me. I've already built a few toy systems that I enjoyed doing, in a way I haven't with "hey there's a new version of grok" or whatever, in a very long time.