calebkaiser
I'm not aware of any of those products really sticking around.
As an example outside of software engineering, a lot of people in marketing roles have found agents to be incredibly useful for particular tasks. I've talked to multiple people who wouldn't standardly be classified as "technical hires" who are using Claude to manage things like digital advertising. It's valuable here because managing campaigns on a platform like Google or LinkedIn involves a ton of granular tweaking and analysis. Every performance marketer makes tradeoffs around what they're realistically able to spend time doing, and what makes the biggest impact. So if you have 250 campaigns that you manage solo, you likely don't have the time to really manually optimize the bidding and budget for each campaign every couple of days based on new data. You can only try so many new angles, because creating new media is time consuming. You don't have time to construct custom experiences on-page for every particular campaign. There's a whole industry of tools that turn a tidy profit automating parts of this for this reason. And agents are really good at it, because a lot of it is just applying the same analysis over and over again.
But for a lot of startups, increasing traffic from ads 20% while decreasing spend is a good outcome, but it doesn't actually turn into revenue. The product itself might just be limited, and already being discovered by the narrow segment of potential users who find it valuable enough to pay. Or the sales process might be kind of busted, meaning the team will really only close a certain volume of new deals in a given month, no matter how much inbound they have. Or some combination of all of the above.
If it's just one marketer using an agent to optimize their advertising, there's not a lot they can do about the other constraints. And so what you end up with is one employee who feels really productive and sings the praises of Claude, but not a meaningful lift in the higher level outcomes you care about, and the analysis gets messy. In an ideal world, everyone realizes where the other bottlenecks are, but that's a genuinely hard thing to do and people have complicating incentives.
I don't think this is a particularly new problem. It feels identical to any other situation where a department asks for a tool, loves it, and it doesn't really make a difference to the bottom line or make anyone else's life easier in noticeable ways.
- Groups focused strictly on model capability (i.e. benchmark scores)
- Groups really focused on governance, interpretability, control planes, etc
- A small but excitable minority of weirdos (me included) who think they're the same thing
A lot of the stuff that gets loosely clustered under neurosymbolic methods might be interesting if you're looking to dive in. The sort of pattern you're describing is something that a lot of people have taken stabs at from different directions, or at least have tangentially nipped at while working on related problems.
One of the most well studied genetic risk factors for addiction vulnerability is the DRD2 gene, which encodes the dopamine receptor D_2. People who are particularly vulnerable to addiction commonly have a deficiency here, meaning getting bored very easily and feeling under stimulated/rewarded by typical things is in part what makes riskier sources of pleasure (i.e. drugs) so appealing.
Disclaimer: Not a medical professional, just like reading about addiction and genetics.
But I also have found that where you work and what AI usage is like in your company is a pretty big deciding factor. Your comment, which I agree with, made me think about this. I consult a decent amount, and I've worked with companies where the engineering culture has adopted AI in a way I describe as "ugly." I should find a better word. But what I mean is that it's sort of like the attitude is "okay, now we can ship more tickets, so everyone take on 8x the workload". And they do a lot of "best practice" kind of stuff around using coding agents--every new anthropic article gets posted to slack, and there's always a new CI process for using agents--but they're fundamentally just trying to do the same work as before but kinda sorta faster. It's a weirdly miserable experience.
On the other hand, I've worked with companies where the engineering culture almost feels exuberant. I think a consistent difference is that in these companies, it feels like the engineers now have permission to try wildly ambitious things. It's not always optimally productive in the sense that more things get thrown away or end up just not really working out, but overall they move significantly faster and do more interesting work. Commonly, it's things they wanted to do before, but simply weren't realistic with their constraints around capacity. Or it's an idea like "this part of the codebase has always been a performance nightmare, but it's also a beast to untangle. I wonder if I could setup an autoresearch loop and let it run all day, even just to harvest some ideas?"
But yeah, when everyone sort of gives up and/or are forced to move an ungodly number of smaller tickets, it is a miserable time.
One of the big points is that looped architectures don't necessarily remove CoT style reasoning. They definitely change it, but the model can still be prompted for this kind of output. It just spends more computation on individual tokens. I think the hope, in that particular scenario, is that the model spends less tokens on things like backtracking.
But there are lots of approaches to looped transformer architectures. Some have some kind dynamic loop (i.e. they might loop until they satisfy some kind of equilibrium, or they might have some kind of routing function that makes different tokens loop differently, or any number of other approaches). Others are simpler "loop this many times" architectures. It's a really active area of research with some pretty deep roots, if you want to dig in! This paper blew up a while ago, and it's a looped transformer architecture (though not like the ones discussed here) that isn't autoregressive, as an example: https://arxiv.org/html/2510.04871v1
But the short version is that there are lots of people who want to buy any given drug and drug dealers sell them. I don't know exactly what you mean by "WAIT for people to demand the more profitable drugs", but there typically isn't a whole lot of waiting. There's a lot of demand out there. Giving away a bunch of free fentanyl to random people in the hopes that one of them instantly gets addicted would be a pretty wasteful and high risk sales tactic, especially given the number of people that want to buy it already.
This is a weirdly melodramatic analogy.
If you want to flatten everything to "all things habit forming are definitionally equivalent", you'll get some pretty funky results.
Turns out that drug dealers by and large prefer the straightforward approach of selling drugs to the huge market of people who want to take them.
And we're talking about subscription plans from OpenAI/Anthropic here. Switching to a different model and/or harness if the price changes isn't a "the world has run out of opioids and I'm going to need a detox center" level crisis.
1. In most cases, like this one, the product isn't heroin.
2. Drug dealers don't actually do that.
My experience in the publishing world is that there is already an infinite supply of writing. Lots of things serve as filters, some for better and some for worse, but ultimately the quality of the writing (however we define that) vs. the field is part of that filter. There is a competition for our limited little attention spans and finite waking hours, and writing better suited to our tastes tends to win out.
I don't see how that dynamic changes with AI. If all writers have more or less equal access to AI, some writers will use it in ways that result in writing I enjoy more than others. Some won't use it at all and still write work I really like. The majority of people writing with or without AI, just as today, won't write stuff I really want to read.
I don't know if I've ever read a piece of SEO-driven content marketing and thought it was particularly profound or that it made an impact on me. Or that I really had an inherently meaningful connection to the author. I sort of don't care if people generate 100x more of that sort of thing. It makes Google searches less effective, I guess? Or it may make my For You Page on Instagram less human-rich?
I've read some nice poetry though. I usually go looking for it. There are a handful of authors I tend to like consistently. The overall volume of poetry published (it's quite a lot!) doesn't really have an effect on my preference for their work. If there was 1,000,000 times more poetry published because of AI, I don't think it would prevent me from seeking out new poems from writers/publishers/curators I like, nor would I probably really notice.
Maybe the Go plan is changing that for others?
I don't think the author is "wrong" about the gap between theoretical discussions computation and the actual design constraints of building usable computers. I don't know if that's really a general refutation of proving new things mathematically though. The word "computer" doesn't appear once in the paper OpenAI released, nor do they make any claims specifically about "computer science": https://github.com/openai/math/blob/main/preprints/An-explic...
It's a novel bit of math. Whether or not it means anything in the context of writing software realistically is an interesting conversation, but it doesn't have much bearing on the substance of the result itself. OpenAI is pretty explicit that they view these as research results in the domain of mathematics.
If we literally mean AI as in people paying to use/deploy AI via Google, that's still a small part of their revenue, far less than advertising revenue. If we mean AI as in "every Google product that uses ML", then I guess that's been their core product for a long time.
People sort of generally refer to the Bitter Lesson as if its key argument is "Large Transformer models are best", but this is directly from the essay:
"One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning."
The second paragraph of the essay is specifically about chess:
"In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. At the time, this was looked upon with dismay by the majority of computer-chess researchers who had pursued methods that leveraged human understanding of the special structure of chess. When a simpler, search-based approach with special hardware and software proved vastly more effective, these human-knowledge-based chess researchers were not good losers. They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. These researchers wanted methods based on human input to win and were disappointed when they did not."
The "brute force search" he's talking about (Deep Blue) used tree search with alpha-beta pruning on top of a ton of human crafted heuristics and a database of game-specific information it could access. Stockfish has historically been an even more "brute force" approach in that it is much more focused on optimizing search--which is one of "arbitrarily scalable" techniques listed in the Bitter Lesson essay--and it has increasingly replaced handwritten human heuristics with neural networks over the life of its development.
At no point in the Bitter Lesson is there a mention of LLMs driving cars, or a suggestion that large transformer models will be the dominant class of model for every single task. It is specifically reflecting on the fact that approaches that leverage search and learning, combined with large amounts of compute, consistently prove to be most effective. Stockfish would qualify as evidence of that, under his criteria.
At the most abstract level, you can observe patterns in a model's activations and theorize that particular collections of components contribute to particular kinds of operations. There are many flavors of this intuition. In that sense, you can think of there being "submodels" within a model, insofar as different parts of the network can participate in specialized computations. Some reading that might interest you in this area would include things like mechanistic interpretability.
There are also Mixture of Expert models, which are a dominant architecture for LLMs now, in which you essentially have a bunch of smaller FFNs that get routed over/aggregated together when the LLM performs a forward pass. It would be a mistake to treat any single expert as a better task specific model though. They are trained as part of an ensemble, and they're almost always being combined with other experts in the forward pass, so they're more like (to use a loose analogy) regions of the brain that tend to correspond more towards particular actions than others. You couldn't just extract one and say "This is now our Chess model that performs as well as the larger LLM we took it from". You could potentially use the relevant experts as a basis for training a smaller task-specific model, and there is work around expert pruning and other distillation techniques, but it's not as simple as "This smaller inner model is a complete task-specific model as is". And more importantly, that ensemble of experts *is* the LLM you're comparing it to, at least in major part. It doesn't really make sense to treat the LLM as some foreign model that calls the expert. The experts themselves don't often correlate to easily interpretable task domains, as they've been learned via joint computation with the other experts and broader LLM.
Then there's tool calling. Today LLMs are trained to make tool calls (basically execute functions) for particular tasks. Those tools can be smaller specialized models, or deterministic code, or anything really that can be fired from a host environment.
A classic paper on "expanding" a neural network (the second part of your distil-then-grow loop) is Net2Net. Their goal is basically to get a smaller model's knowledge transferred into a larger one to bootstrap the larger model training: https://arxiv.org/abs/1511.05641 . Bert2Bert is basically the same idea but language models: https://arxiv.org/abs/2110.07143
There's also stuff on "growing" a pretrained LLM that is somewhat similar to what you're suggesting: https://arxiv.org/abs/2303.00980
And then there's a bunch of stuff on "refreshing" weights in the network as well, sort of like pruning but without reducing capacity.
One of the things to explore are assumptions around how information really compresses down in distillation, and what is really happening when we deem a model saturated. There's often an intuition that when you distill the model down, you're finding a sort of "truer", more essential representation of the functions you're modeling. And so if you distill it down and then expand the weights, you've added additional capacity for it to learn more, since the distilled core now contains the important information from the original larger model. But very often that's not actually how it plays out, and the problem is thornier than it seems on its face. Plateaus in training don't necessarily mean the model has run out of representational capacity. And distillation is not guaranteed to preserve the most valuable information. It's entirely possible that the representations the original model has learned for performing on the tasks it is evaluated against are more generalizable than the compressed representations you've distilled.
Initialization is also tricky when you're expanding the weights like this. Non-linearities make it such that splitting the weights directly like you're describing doesn't preserve the function. Not that this means the model can't recover from further training, but it's just not where you want to start training from if you can avoid it. There's also a strong likelihood that the added parameters are redundant relative to the distilled core, which will make learning more challenging. A lot of work in this space as a result focuses on stuff like symmetry breaking. A cool paper you might find interesting that isn't about LLMs, but has a similar "Progressively give the model more degrees of freedom" flavor is ARC-AGI Without Pretraining: https://iliao2345.github.io/blog_posts/arc_agi_without_pretr...
But there's lots of active research around this general space. You should test out your ideas and share them! I can't recall now exactly but I feel like I've seen MoE stuff that touches on some similar ideas.
But what I do use Pi a lot for is as a base for agents. I much prefer it to using an agent SDK. I find its minimalism and extensibility to be a really nice substrate for new projects.
I could easily imagine someone getting to a really productive personal setup with it as well, for the same reasons as above.
In general, training a general purpose classifier is something lots of people have worked on for a long time. Large Transformer models themselves are typically "generalists" already, so structured generation and constrained decoding have given you the ability to use an LLM as a general classifier for years. It's an incredibly common pattern for working with LLM judges or any sort of branched decision making workflow.
A lot of people who are a bit less familiar with the field saw the hype around Jev and presumed that the reason it was so exciting was that it was a fundamentally new interface for working with an LLM. And that additional excitement drove even more attention to Jev. But fundamentally, TypeSafe's announcement was that they found a particular architecture/training paradigm that resulted in a model for this particular interface that had incredible accuracy, very low latency, and for which they could offer inference at a super low cost.
I've not kept up with the flood of Jev clones that have been released, but I think this is just typical for any new component in deep learning that gets popular. There are an absurd number of open source autoregressive LLMs and fine tunes you can use. The thing that makes one more popular than the other is typically the general performance of the individual model.
But training a model for this purpose, or emulating the procedures described in Jev's papers, isn't something that would be beyond the capabilities of any lab. It's not an entirely alien architecture or approach.
The bigger question for TypeSafe as a company would be if other teams are producing Jev-like models that win on performance or cost. Like I said, I haven't followed the reports super closely, so no idea if that's the case or not.
At a high level, agent sandboxes have peculiar needs. Agents are really bursty, but also long lived. You need low latency suspend/resume calls. Checkpointing and recovery have some particular considerations. And naive approaches are often really wasteful, but over optimizing without harming durability, isolation, or consistency in performance can get tricky.
This is the new hot infra topic for agent swarms, for the time being. Whether that's impressive or not is up to the reader I guess, but it's a pretty involved project nonetheless.
In early 2016, Clarifai's core product was basically just an image classifer exposed via an API. And at that point, they'd raised $40 million--the same amount as TypeSafe.ai/Jev--and they were experiencing viral growth among developers + signing contracts with a bunch of flashy logos. The demand was so high that Amazon launched Rekognition and Google launched their similar APIs to compete.
The AI hype cycle and the number of people thinking about using AI certainly puts more wind at Jev's back, but even in an alternative universe where we don't have contemporary LLMs, Jev's core claims would be remarkable and there would be a big market for it.
I think the hype with Jev is just that, while structured generation is great, LLM judges tend to kind of suck for precise classification. And the more powerful the base model, the more accurate they can get, but they get increasingly expensive/impossible to finetune. It was specifically the latency/price point Jev offered vs. the general accuracy it claimed that generated all the excitement. Plus the promise of cheap calibration (tuning).
"Jev exists because LLMs exist" is kind of a truism, as Jev apparently is literally a Transformer model.
In addition to the ones you listed, I'd add the V8 runtime, Jax, Protobuf. Even some of their projects that wound up declining in market share (Angular, Tensorflow--both losing share to projects that wound up at Meta, ironically) are still actively maintained and pushed.
But I'm sure there's also a huge graveyard of open source projects they abandoned that just never hit my radar. Still, at least with their open source stuff, you can fork in the worst case.
But I've spoken with many people at companies who've decided to add an in-UI agent to their apps, and I don't think this trend will persist. Absolutely AI/agents will increasingly become a major component of interfaces, but the current common incarnation of "Clippy for X" feels like a kludged bridge between an app that was not designed from first principles for agents (and often was poorly designed for humans) and the impulse to be "AI native".
From my experience, some of the most successful niches for this sort of UI so far have been in apps that were already well suited to it. I'm thinking specifically about analytics/dashboarding/"this is a portal for you to query things easily" software. They already started with a lot of the elements you want: visible provenance of the agents actions via the queries it writes, a malleable interface that you already expect to be customizable and ephemeral, and most importantly, "navigation" that is genuinely difficult for many users (in the sense that "navigating" can mean "querying specific data"). The agent provides a ton of value to users and its actions are intuitive and legible.
But the agents you see right now in a lot of apps that do things like navigate you to the right page by... sending you a link, which you could have clicked from the navbar? Or worse, which hijack your navigation and throw you on a page you're unfamiliar with, and where you have no sense of place or how to make your way to/from? I don't think they're particularly long for this world.
I agree with you. I think the OPs pushback is emblematic of a larger reaction I've seen that is, at the very least, misinformed.
There are a lot of approaches that use a self-attention backbone for classifier-style outputs. You have structured generation libraries like SGLang and Outlines, but those basically give you guided generation on an autoregressive model. You also have a bunch of models that are non-autoregressive that try something similar. Older NLP stuff applies here, and there's newer stuff using diffusion transformers for this purpose.
But I don't think the Jev author has ever said that he's the sole human, alone in a vast sea of misguided researchers, who is interested in schema-guided classification? I think he said he found a novel way to train a model for this task that has much higher general intelligence at much lower cost than other approaches. Which is an exciting result with lots of applications if it bears out.
I think some people are just reflexively skeptical of anything that gets a lot of hype. Maybe that's fair. Things that are wildly successful and high impact also tend to get a lot of hype though, so it seems like a poor filter.