I think the better unit to commit and work with is the prompt itself, and I think that the prompt is the thing that should be PR'd at this point, because ultimately the spec is what's important.
The fundamental problem there is the code generation step is non-deterministic. You might make a two sentence change to the prompt to fix a bug and the generation introduces two more. Generate again and everything is fine. Way too much uncertainty to have confidence in that approach.
Also, people aren't actually reading through most of the code that is generated or merged, so if there's a fear of deploying buggy code generated by AI, then I assure you that's already happening. A lot.
That said it's so trivial to do, why haven't you done that already?
But more generally, I actually have been building some CI stuff to automate how I'm saying.
I don't have much of a say how this is handled at work so they're just committing the generated code, but I actually am doing what I am talking about.
> I have been writing a library called "assume" where you can specify a type signature, give it a prompt, and it generates a function on the fly in the background with Claude Code, so you still write some code, but whenever you need a function you "assume" that such a function exists.
[1] https://mail.haskell.org/pipermail/haskell/2005-December/017...
And this is why I boldly compare current LLM craze to the much less hyped craze of strong type systems. I was a part of that strong type system discussion, advocating for them. ;)
LLMs are neat, code generation is neat, but I do wish that people had learned type theory and instead used those.
I'm not aware of djinn, but I do remember the "Type Driven Development" that Idris had that I thought was absurdly cool; when you make the type specification clear enough, there ends up being basically exactly one reasonable way of writing the code, in which case it can just be "deduced" by machinery.
I'm a huge advocate for formal methods, and it does sort of bother me that pretty much all work on that seems to have been refocused on AI.
> If you make the prompts specific enough and provide tests that it has to run before it passes, then it should be fairly close to deterministic.
> I assure you that's already happening. A lot.
Absolutely. The difference is that the amount of bad code that could be generated had an upper limit on it — how fast a human can type it out. With LLMs bad code can be shat out at warp speed.