Anyone trusting it to just "do its own thing" is out if their mind
I think if someone's goal was just the tutorial code, it would have been very impressive to them the AI can summon it.
I'm too poor for local llms, I think there might be a 2 or 4gb graphics card in one of my junk pcs but thats about it lol
Actual novel stuff is so far out on the long tail of iterations that it's a gamble: it might pop up in an early run, or might take 2000 prompts and $20,000 worth of tokens. And it's still not really engineered, it's 10,000 monkeys with typewriters copying random shakespeare snippets off the chalkboard. At some point you'll get all of Hamlet, but most of the time you'll get garbage, and sometimes you'll get Romeo & The Taming of The Tempest.
The other day though I asked for something simple and it one-shotted the problem. To me, that's new.
I know this success was a statistical outlier, however. I grok how to use it and to not trust it. I'm just shocked so many people smart people fail to understand it.
AI has led us into a deep spaghetti hole in one product where it was allowed free rein. But when applied to localised contexts. Sort of a class at a time it’s really excellent and productivity explodes.
I mostly use it to type out implementations of individual methods after it has suggested interfaces that I modify by hand. Then it writes the tests for me too very quickly.
As soon as you let it do more though, it will invariably tie itself into a knot - all the while confidently ascertaining that it knows what it’s doing.