Example of a Claude Code session after 2 hours of "Crunching" that came out without results https://github.com/mohsen1/tsz/pull/4868
Funny thing is, the last percent of the test have been so hard to work on that Opus 4.7 routinely bails and says "it's too involved or complicated" so I had to add prompts specifically asking it not to bail.
GPT for instance had a lot of issues using git worktrees, and didn't understand how to correctly use it to then merge stuff back into a main branch, vs Claude which seems to do this much more naturally.
GPT also left me with broken tests/code that I had to iterate on manually, Claude is much better about reasoning through code. Primarily Python.
I wonder how much of that is due to the model being somehow better, or the harness having built-in instructions on how to use them.
I've used worktrees with Codex just fine, but I instructed it to use my scripts for setting it up and tearing it down. The scripts also reflinked existing compilation artifacts to speed up compiling and allocated a fresh db instance for it, but then also applied a simple protocol for locking the master repository during merges, so multiple agents wouldn't try to merge at the same time. It has been following those instructions quite well.
Pretty impressive that it is faster than the Go version already.
It's much faster in single file benchmarks (3 to 5x)
https://tsz.dev/benchmarks/micro
I have optimizations planned for large projects that I'm still flushing out.
In Nova JavaScript engine[1] I've done exactly as you've done and split objects into typed side pools (I call them "(typed) heap vectors") but in a JavaScript engine my _hypothesis_ is that the visitation patterns are much more amenable to this: an Array, Set, or Map is more likely to be homogeneous than heterogeneous, and therefore a loop over the contents is likely going to hit the same side pool for each entry.
[0]: https://www.youtube.com/watch?v=s_1OG9GwyOw [1]: https://trynova.dev/
Now that most of the implementation is nearly completed, I'm building a lot of instrumentations to have better visibility into those things to have concrete answers to that sort of question. I'm experiencing huge RSS right now that could be exactly what you're pointing to.
It could be another marketing stunt like Mythos, which is so dangerous to release that Antrhopic must be bailed out by the government.
We don't know if the timeline is true, whether Rust experts had a hand in it or even if the reported test suite compliance is true. We are dealing with a company of habitual liars and promoters.
The branch is open.
You can check it out and run the tests if you don’t believe it.
Any sources to back this up?
I question this. Yes, strong enforcement of invariants at compile time helps the LLM generate functional code since it gets rapid feedback and retraces as opposed to generating buggy code that fails at runtime in edge cases.
On the other hand, Rust is a complex language prone to refactoring avalanches, where a small change in a component forces refactoring distant code. If the initial architecture is bad or lacking, growing the code base incrementally as LLMs typically do will tend towards spaghettification. So I fear a program that compiles and even runs ok, but no longer human readable or maintainable.
This may be so, but LLMs are great at slogging through such tedious repercussions.
I would say if the language prevents sloppy intermediate states, that actually makes it more amenable to AI; if you just half-ass a refactor into a conceptually inconsistent state, it’s possible for bad tests to fail to catch it in Python, say. But if many such incomplete states are just forbidden, then the compiler errors provide a clean objective function that the LLM can keep iterating on.
Are you saying this out of personal experience or just hypothesizing? I am working on a large, complex rust project with Claude Code and do not experience this at all.
- write sleek operator-overloading-based code for simple mathematical operations on your custom pet algebra
- decide that you want to turn it into an autograd library [0]
- realise that you now need either `RefCell` for interior mutability, or arenas to save the computation graph and local gradients
- realise that `RefCell` puts borrow checks on the runtime path and can panic if you get aliasing wrong
- realise that plain arenas cannot use your sleek operator-overloaded expressions, since `a + b` has no access to the arena, so you need to rewrite them as `tape.sum(node_a, node_b)`
- cry
This was my introduction to why you kinda need to know what you will end up building with Rust, or suffer the cascade refactors. In Python, for example, this issue mostly wouldn't happen, since objects are already reference-like, so the tape/graph can stay implicit and you just chug along.
I still prefer Rust, just that these refactor cascades will happen. But they are mechanically doable, because you just need to 'break' one type, and let an LLM correct the fallout errors surfaced by the compiler till you reach a consistent new ownership model, and I suppose this is common enough that LLM saw it being done hundreds of times, haha.
However, a) I think the compiler telling me everywhere I need to fix it is great, and b) even before LLMs, using compile mode with emacs and setting up a macro to jump to next clippy warning, jump to code, fix, and then repeating in batches of like 20-50 could often make it go quite fast.
I've come away feeling that most it looks fixable - but it won't be fixed in Rust. Some of the language choices (like favouring monomorphization to the point of making dll's near impossible) are near impossible to undo now, and in other cases where it might conceivably be fixed (like async) it won't be because the community is too invested with their current solution.
So we are stuck with the Rust we have; warts and all. That blog post convinced me those warts mean the language should be avoided for game development. Similarly sqlite developers convinced me the current state of Rust tooling meant it wasn't a good fit for their style of high reliability coding, so they are sticking with C. Which is a downright perverse outcome.
But for most of us C programmers who aren't willing to put in the huge effort Sqlite does to get the reliability up, Rust is the only game in town right now. It's the first and currently only language to implement a usable formal proof checker that eliminates most of the serious footguns in C and C++. But I am now hoping it becomes a victim of the old engineering adage: plan to throw the first one away, because you will anyway.
I haven't had any issues with this getting out of hand on >10KLOC vibed rust codebases.
This rewrite is >750k lines of Rust
The lesson here is that right now LLMs are a lot better at "fill in the implementation for this API I defined" than "design everything from scratch" if you care at all about whether it becomes a mess of spaghetti. Maybe someday they'll be better at it, but at least today, you have to choose between going full vibes and not caring about the code, or you need to be involved in the design, and either way it's not clear that Rust is a significantly worse choice based on anything other than your own experience.
I've also seen the benefits of Rust for this too. And making the bet that my pg experience will help me make good design choices around many of the things people have been having trouble with in pg for a long time[1]. Excited to see AI make it more possible to improve complex pieces of software than has historically been practical.
[0] https://github.com/malisper/pgrust [1] https://malisper.me/the-four-horsemen-behind-thousands-of-po...
curious about your workflow for running all these accounts. different harnesses in parallel? manually switching in codex? 5.5pro only?
what works for you?
[0] https://malisper.me/pgrust-rebuilding-postgres-in-rust-with-... [1] https://malisper.me/pgrust-update-at-67-postgres-compatibili... [2] https://github.com/loongphy/codex-auth
You use the test suite and LLMs are trained on Postgres.
Are you at Freshpaint? A company that "helps healthcare marketing teams grow in a world where privacy is the baseline, but performance is the goal."
Nice promises! Surely the marketing teams will respect privacy!
Left to itself, it often follows human developers who conceive of their goal as "get the program working, the end justifies the means." Which makes sense because there are a lot of systems like that in the training corpus.
Rust is a terrible language for using LLMs to write code if Rust's low latency isn't needed, because of its extreme compile times. LLMs code faster than humans so a far bigger fraction of the time is spent waiting for the compiler, and a reasonably sized project will take literally 10x longer to compile in Rust than in e.g. Zig or Go.
Also remember, `cargo check` is quite fast, and wholly sufficient for confirming correctness.
100%. I've been telling everyone who will listen this for 2 years. LLMs are infinitely more productive with swift code like
let engineCycleCount: Int = 5
vs
let eC = 5
They still make mistakes, but forcing _explicit_ typing in a strongly typed language makes them make far fewer mistakes + the compiler is catching >90% of what you try to catch with a billion rspecs in trash languages like ruby.
But if you mean in general, I also totally feel that languages that let you represent more invariants statically are better fit for LLMs. I'd love to see experimentation with LLMs with dependent types and managed effects.
https://tsz.dev
Rust is perfect for writing all of code using LLM. It's strict type system makes is less likely to make very dumb mistakes that other languages might allow.
Also want to note that writing the code using LLM doesn't remove the need to have a vision for the design and tradeoffs you make as you build a project. So Jarred and his team are the right kind of people to be able to leverage LLMs to write huge amounts of code.