mccoyb

Why do people keep calling this a paper?

It's a blog post which received feedback from collaborators.

It hasn't gone through the academic paper lifecycle ... the authors themselves refer to it as a post and as preliminary.

I feel like I'm taking crazy pills, but can we be precise when talking about work of this form? It started with referring to arXiv pre-prints as papers, and now we're at blog posts with an academic template as papers.

A paper is a fucking paper: something which has been vetted by the scientific community, with at least some form (often, a few rounds) of anonymous review from people outside your circle of collaborators

This is not there yet, and clearly so due to the various feedback from people (including the two pushback posts on HN) critiquing the lack of discussion about the existence of these techniques in mech interp, etc?

Right, makes sense — yes, all good questions that I’d also like to understand the answers to.

I think you're misunderstanding the premise.

The binary is running. Maxwell is watching it: presumably there is some cost to the watching, but the LLM is not running in the binary process. Maxwell also has dynamic information about the running environment. Maxwell can then try and superoptimize the running binary using the dynamic information it has, but to do so, it needs to create semantics-preserving patches to the running binary.

What should be clear here: dynamic information is strictly more information than what a static AOT compiler has access to.

So that's the problem: we're going to patch the binary while it is running, and we need the patches to preserve the semantics as expressed by the high-level program, possibly assuming that the compiler was semantics preserving, etc. I'm assuming they have custom compilers or something for all of this.

Getting this to work probably involves creating some sort of source map, perhaps a custom linker, etc (just guessing) from high-level program to binary -- but assuming you handle all of that, then the job of Maxwell is just the job of a normal superoptimizer: propose semantics-preserving patches, see how they do, etc.

In theory, you could propose terrible patches frequently - superoptimization normally discards these, but you with bad proposals you might spend a long time searching for good patches. But LLMs are good proposers now, so you'd probably mostly get good patches (to a limit).

Authors, correct me if I'm wrong.

I agree with you that Maxwell is like a JIT (with superoptimization inside of it: a very smart one which can propose smart things using the underlying agentic model, and reason about semantic equivalences to figure out what to commit or not) and that that is a natural analogy.

But people who write JITs don't call the overall process (original code, JIT) "self-improving code", do they?

What is the "self" in (original code, JIT)? Is it the entire process? But the JIT doesn't make itself better as it runs? Is all code that runs in a managed runtime with dynamic analysis self-improving?

A much better term might be "dynamically-improving code"

But related and to the point of my original message: the phrase "self-improving" has been used in vogue by AI research / labs ... and this is another example of something which is not a clear term, and seems intended for marketing

Let me say that I like the idea:

- verification

- superoptimization specialized to a particular piece of hardware

- patching binaries while they are running

all great

That being said, I'm getting some weird vibes from the prose on this site. Little phrases and claims that seem intended for the purposes of marketing, but conflate things with clear technical meaning in confusing ways.

For instance: I watched the video ("a code execution model has to be right every time") which implicitly communicates something like "the model cannot hallucinate" ... of course, the agentic LM is still probabilistic, but it is wrapped with verification tools (to reason about semantic equivalence), and is likely post-trained to be good at using those tools.

This strikes me as a conflation between the model and the harness which is intended to strike a marketing point (like: "it doesn't hallucinate") but which is clearly a ridiculous claim to make (falling down the slope: any harness which does error correction for improper tool calls might also be a "model which makes no mistakes")

Also: "self-improving code"

that's not really it, is it? Self-improving code would be code that runs itself and improves itself. This is code I run, which Maxwell watches and improves ... no?

Related: I dislike the usage of the word "model" in the above discussions on the site. Do we want to call everything a "model" now? Can anyone invent a new frontier "model" by post training an agentic LM with tools?

Also, let's say you are optimizing kernels on GPUs. This can be subtly fraught: for instance, it was well-known for awhile that one should only use certain pieces of software (vLLM, for instance) to reproduce papers (which used vLLM) there are non-deterministic sources of randomness (including, of course, floating point) that would prevent one from recovering the performance claimed in a paper ...

See, for instance: https://news.ycombinator.com/item?id=49182996 by Kenton Varda

Also, not mentioned in my post:

- Mitchell Hashimoto

- Prime Intellect (and all their agent experiments)

- Geoff Huntley (see Jiti, for instance)

(many more)

There's a ton of interesting software being developed with these models by people outside the in group, but I find most of the software from these big guys to be ... bland. Buggy copies of copies.

No, I’m talking about the recent DevDay.

Also, yes there are still bugs in Claude Code. I experience them nearly everyday.

It is markedly better than early days, but still not the best harness.

The best software written with agents seems to come from people outside of the labs (see pi, for instance — or all of cloudflare’s recent work)

Which makes me question either the model, or the holders …

If the software coming out of OpenAI and Anthropic is what we have to judge, I wonder about the 5000 ...

Let's say, for the sake of argument, that the models are some multiplicative factor better on the inside.

Doesn't that mean the demos should work?

There's several things I disagree with in this post (a matter of perspective, I think):

> The discourse around software engineering has gotten dumber. It’s like everyone in the industry lost 30 IQ points. People used to talk about compilers, type systems, logic. Now they talk about “prompts”, “harnesses”, “loops”. The discourse is narrower, shallower, and more repetitive. There’s only so many times I can hear about “agentic harnesses” before I lose my mind.

I agree with the frustration here :)

> You design a revolutionary new programming language: who cares? Maybe Claude cares, for what that’s worth. But humans don’t write or even read the code anymore. The programming language is an implementation detail the humans no longer have to care about.

Perhaps the people who weren't inclined to think about compilers, type systems, and logic before coding agents will never think about those things. If anything, I think about those systems significantly more now, because I view them as ways to describe the search space for the agent ... and perhaps, in the future, even tools to structure agentic latent representations.

So I propose rotating your viewport: yes, perhaps entertain the idea that it doesn't make sense to think about these things for human consumption ... but instead, what if we think of programming language and compiler design as being a tool to control the search space, and suddenly there's lots of new ways to think about these things.

Is this a valuable way to think? I'm not sure, but we should try and find out.

Call me crazy, but if you can't explain the idea in 10 pages or less, you don't understand the idea.

(For instance, 58 pages, 80 pages ... I don't think of these as normal academic length if this is the main body content ...)

Are you against accessing hardware that you own?

I love these commercials where someone has chosen to show how the AI product will essentially be used to slop out some garbage piece of corpo communication ... and the user basically says "looks good" with barely any thought, then mixed with some sort of real life thing (wedding planning here).

I mean, I feel like I'm going crazy -- but I was struck by this jarring blending of experiences ... shitting out some growth plots followed by autopilot on your wedding. Nice OpenAI. The only thing missing is a moment of self-reflection where I contemplate where exactly I lost what makes me ... me.

Is this what SV wants the world to look like? Mixing fucking cake batter while a bot shows me a regression to the mean website? Pretending like I have any sort of intentionality in my life, while a nameless entity (given quirky form) sort of walks me through my life?

I'm not sure why it gave me this impression, but strikes me as vaguely reminiscent of soma (from Brave New World).

Kind of sad, because the tech is actually incredible: who are they hiring to storyboard these commercials?

How do you know that "we" in "we do not use facial recognition technology" covers ambient.ai, who presumably gets the feeds directly?

Agree with you, but not sure what form the arms race will take as the months go by (between harness engineering, RLing models within harnesses, etc)

Oh I agree with you! I just don't see "we're standing on the shoulders of giants" in most of these marketing blog posts?

If I'm wrong here, I'd love reference links. I think of these companies as trying to inspire the idea that Claude (or GPT) are these special alien entities, in a sense?

Notwithstanding the duplication of these posts across social media, OpenAI, and Anthropic ("it's all the model, they just tell it to keep going" if you beat anything with RL enough ... it's going to do the thing)

Here's my issue with this post:

> True to the spirit of the challenge, they didn’t use millions of dollars in computer power. They used Fable 5.1, working within Claude Science, a platform scientists can pay to use.

Okay, billions of dollars have been poured into these agentic LMs, right? Each training run to get the next increment is costing millions of dollars?

This feels like an obvious jab at Navier-Stokes, but where we get to shift the numbers around to hide where the compute actually is being spent ... compute is being spent. It's either being spent in amortization to make the search smarter ahead of time, during training, or its being spent after.

Also love: scientists get to pay Anthropic to work within their special science harness to do science. That's exactly what I dreamed of doing when I pursued physics in undergrad, one or two companies holding the keys to "progress" for a monthly subscription price.

I'm always surprised by statements about "things we don't need to learn anymore" with AI.

It's sort of a litmus test for how little someone understands how learning in humans works or AI itself even. "Agentic" AI acts as a mirror: the more insight have yourself, the more powerfully you can pull the distribution ... the more you get out of it. (there's a bloodborne-inspired parable in here somewhere ...)

I think it's why the software from AI companies has "garbage" qualities. They are assuming that AI is going to fill in skills ... when it is a reflective amplifier.

But back to the discussion of "things we don't need to learn anymore": do we go to the gym, and say ... oh I don't need to do pull ups anymore, because there's an assisted pull up machine and I can just set it to my weight ...

Utterly idiotic.

Here's the one simple trick that fixes everything:

- Use the code that your agents write in anger.

There you go. Do I know when my agents fuck up? Yes, I absolutely do -- because I'm a user of the code I have my agents write, and I ask things like "why is it taking 50 ms to start this program ..." and then I go in and find stupidity, and excise it. I do this over and over again.

Is it faster than writing it out by hand? Maybe! It's definitely a different perspective.

Start behaving like a baby "why, why, why" and then do a bit of reading, and you'll be fine.

A lot of these blog posts seem like they're aimed at software written by B2B companies who don't even use their own software ...

I think someone can follow "good practices" with a team and get to a bad place, especially with such a novel product (as AI coding agents).

But taking Claude Code as the product of this style of thinking -- who is Claude Code for? Is it for everyone in the world? Well, if you look at the feature velocity, it seems like the answer is intended to be yes ... Claude Code is trying to solve every problem in software development in the world, all at the same time.

So I question the "user model" here.

Here's another thing that is true about Claude Code: it's among the most inconsistent and buggy pieces of software I've ever encountered.

- You can move the cursor with the mouse in the composer, but not in AskUserQuestion?

- When agents spawn subagents, the model name is inherited from the main agent, and seemingly none of the (4! yes, 4!) subagent tools seem to get this right (except for Explore, which seems to be fixed to a weaker model)

- Sometimes, when my usage limit halts, my agents will pick up when it refreshes (within ~2 hours or something) ... other times, nope -- even within the usage limit?

This is a sampling of my own experiences using this thing frequently. Are these sorts of details not important? Maybe not: I'm not at the level of this team, and may never be.

But I think it's a reflection of agentic engineering ... a somewhat embarrassing one, from my perspective. It paints a picture of a team who can't quite get the details right, even with the assistance of purported extremely powerful AI tools, even internal ones which we don't have access to?

I think when people look back on 2025 -- Boris is going to have his name right there in the books ... Claude Code, coding agents -- Anthropic (& Boris + team) made the first move.

But now it's 2026, and people know how harnesses work, and heavy lies the crown.

Perhaps because their concerns are different? Similar reasoning: "why invest in dependently typed languages over SMT solvers? SMT solvers are automatic, and it is way easier to add annotations to my programs"

The answer is that these technologies are not universal in trade off space.

Of course, if you can have an LLM prove something by dispatch to an SMT solver ... you want to do that (and, indeed, LLMs do sometimes do this even when working on Lean proofs)

But the space of statements that you can prove using verification with SMT is smaller than dependently typed systems.

Also, Ada/SPARK are very good and robust -- they've been around for ~50 years ... but there's been a significant amount of work in formal verification since then, right? I would be more clear that Ada/SPARK are state-of-the-art for "mission critical software" (and have been for many years), but there's been a lot of research work in formal verification of low level programming since then (Low*, for instance)

Thanks for clarifying. I would like to see a more clear exposition on the relationship here.

@LiamPowell the author is clearly aware of formal verification, they've written several implementations of dependently typed languages, and ... despite the presentation of their work, which has some obvious flaws (as can be judged by reception) ... their many comments indicate that they know what they are talking about.

Your post is setting up a strawman between automatic formal verification and formal verification using interactive theorem provers ... obviously there is a spectrum, and Ada/SPARK are navigating the space to try and automate much of the work required to automatically dispatch with obligations to prove (computable) properties about programs.

Bend2 is a QTT -- it's dependently typed, and comes from the lineage of systems which are focused on being expressive enough to formalize mathematics.

Of course you need to build a somewhat significant "standard library" of theorems, tactics (as metaprograms), etc ... to approach what is built into the compiler in Ada. These are different approaches with different trade offs.

Your post isn't clear, you don't go into any of these details ... why did you post this? Do you think this is clear writing?

My read on this, after ingesting a good amount of content on the history, is:

- this Bend is not really related to the old Bend (only in name)

- this Bend doesn't really have anything to do with interaction combinators

- this Bend is a QTT, with a change to affinity which enforces a good performance property for GPUs

- the "higher order at comptime" is neat, reminds me of Andras Kovacs' work on 2ltt and staging in dependently typed languages.

- this Bend is likely to be good at "balanced recursive computations on ADT", and can parallelize them ... but won't be as good as CUDA or e.g. Futhark on dense rectangular array computations

- performance needs improvement in the scheduler, to possibly help with balanced work (looking at the n queens and symbolic regression numbers)?

How are you going to handle search or synthesis over irregular structures (SupaGen)?

Are you a working mathematician?

AI is extremely useful, but it’s also extremely easy to fool yourself into thinking you understand what is going on without really understanding. This is often true with the code, but also for math and science concepts, etc.

Not many professions are formally trained to be cognizant of this lack of understanding, and how to confront it.

Usage of AI in collaborative settings is an amplifier of these issues, especially if someone doesn’t realize they don’t understand: they couldn’t teach or explain the concepts they use, or be forced to work with them malleably in a way that an expert or researcher would.

If you are cognizant of your lack of understanding, you can remedy it by slowing down and teaching yourself. This is required to make better use of AI in the domain of interest!

But you can’t have all things at once: you can’t move at speed with AI, collaborate effectively, and understand what is going on as an expert would. It is not physically possible for a human brain.

If only our interactions with "non-teaching" languages were anything like this ... or humane, in the lineage of Bret Victor, etc.

Also shout out @tonyg for Syndicate! I keep coming back to it -- a beautiful piece of PL work.

I had nearly the exact same experience and thought I was imagining it … absolutely ripping, then it turned into Sol++ on Tuesday …

I’m working on hard things, it is very noticeable when it is hums through something and then falls over on something it should not

I can tell by analyzing my own prompts to look at when I get frustrated ;)

I hope this is satire.

Did you read or watch any of the post mortems?

No, it is a shocking level of incompetence given the conveyed seriousness of the work by these labs.

So yes, models are getting better. Ask yourself: if you know that to be true, would you act the same way that the teams did in the public post mortems?

Agree with you, overall -- and it's worthwhile to learn how to setup factories at various scales of problems, and one will always be in a situation where you need to decide whether you care about a particular part of the process, or can be okay with averages.

I'm only saying that that I don't think one can automate the whole thing. After all, agents don't actually possess ... agency -- they require us to breath live into them. So at some part of the stack, there has to be a human, with an opinion, thinking about what they want.

Also: I don't aspire to be a worker, I aspire to seize the means of production.