> it often doesn't work at all without numerous revisions?

It doesn’t so for many/most humans either. I hope they have a revisions prompting, but I did not try it yet.

I noticed adding in a feedback/review loop often fixes it, but the you still need someone saying ‘this is it’ as it doesn’t know the cut off point.

I think there's a big gap here you might be missing. Most developers beyond juniors can generally write code that at least compiles on the first pass, even if it isn't functionally correct. Current AI models often generate code that doesn't even compile.

> Most developers beyond juniors can generally write code that at least compiles on the first pass, even if it isn't functionally correct.

Hahaha. I’ve been coding for over 20 years and this is definitely not the case.

> Current AI models often generate code that doesn't even compile.

Most of the code ChatGPT has given me, has run/compiled on the first try. And it’s been a lot longer and complex than what I would have written on a first pass.

Let’s just learn to use these tools instead of trying to justify human superiority.

I've found with my own projects that the AI allows me to focus on the interesting stuff. I was porting some quant work from a Jupyter notebook into a flask app, and the AI is quite handy at providing boilerplate for stuff that isn't relevant to what I want to work on. I need a site that can display charts for a given analysis of an indicator, and ChatGPT handled that for me quite handily.

> Most developers beyond juniors can generally write code that at least compiles on the first pass,

Aha. Maybe you know super clever people or people who learned in the 60-80s when cycles (and reboots etc) mattered or were costly; this incredibly far from the norm.

Is that a problem though? In this IDE the LLM sees the error message and tries to fix it, possibly while the developer who wrote the prompt is off doing something else.

Revisions prompting is not there yet but it is one of things that we are experimenting with.

The feedback/review loop is spot on - a lot of the problems can be fixed automatically in a few steps but you actually need the outputs/errors.

I made one that tries to get to the end code by having 3.5 and 4 play different roles and correcting eachother. Sometimes it works, mostly it loops being unable to get to the end.

Good point and feedback, thank you. We'll update readme.

A lot of UX, UI, DX work related to LLMs is completely new. We ourselves have a lot of new realizations.

> While you mention that you can bring your own model, prompt, etc, the current main use case seems to be integrating with OpenAI

You're right. This is because we started with OpenAI and to be fair it's easiest to use from the DX point of view. We want to make it more open very soon. We probably need more user feedback to learn what would be the best way how this integration should look like.

> How, if at all, do you plan to address the current shortcoming that the code generated by it often doesn't work at all without numerous revisions?

The AI agents work in a self-repairing loop. For example they write code, get back type errors or a stack trace and then try to fix the code. This works pretty well and often a bigger problem is short context window.

We don't think this will replace developers, rather we need to figure out the right feedback loop between the developer and these AI agents. So we expect that developers most likely will edit some code.