The LLMs will need to get a lot better though. I see every day on reddit/X a bunch of people saying they downloaded Cursor and built a saas without ever having coded anything in their lives; the resulting code is always terrible, full with really obvious bugs and buckets of trivial security issues. All these things can be prevented, in theory, but current LLMs really don't do that at all (they aren't capable).

All of these SaaS products are also clones of things people have already made. If it was helping newcomers create novel things from great imaginations and curiosity, I’d be really excited about this. Instead I saw a reminder app, a todo app, a combination of these, and a voice transcription app.

All of these have existed in huge numbers for a decade or close to it. This isn’t interesting. It wouldn’t have taken much to do this without AI if these people found motivation through some other means. Nothing notable is happening yet.

> All of these have existed in huge numbers for a decade or close to it

That doesn't always matter though; I saw some guy on reddit selling 1000s of subs of a server uptime page they generated with an LLM (Cursor I think it was). Most do it for the money and that is working IF you have the following somewhere.

Do you have a link to that?

> the resulting code is always terrible, full with really obvious bugs and buckets of trivial security issues

That's pretty much the same as what you got from old school stackexchange. The LLMs are trained on poor example data (e.g. from stack exchange) and I presume it is a hard problem to filter for just the "good" training data.

Stack exchange answers make up a tiny fraction of LLM training data. They've dumped the source of every open-source project on the internet into those things. They've been trained on plenty of high-quality code; the trouble is that all that can teach is imitation.

The why behind a design decision is something an LLM can't understand simply by being fed the decisions themselves. Hence, though they often stumble into the right answer, they only ever do so because it "seems right based on context," and so they can easily apply a principle correctly one moment and then misuse it the next.

ffmpeg, for one.

Compilers actually output the binary that you ask them to, though.

If all a compiler did was spit out a slurry of buggy assembly that misunderstands its context and has to be carefully scrutinized for errors, I would still be writing assembly the old fashioned way.