That's like me with violins!

Maybe violins don't write good code after all?

I can see that you only work with true scotsmen, but the machine really does write most of the code now, at least from a pure numbers perspective.

Right I feel like they just got to a point where they can reliably write individual functions with direction. Every time I've tried to use them consistently I feel like I just end up spinning wheels from them doing things that mostly work until it crashes and burns or just has straight up missing things or half truths. Skill issue or not it doesn't work for me.

Still glad I basically never have to debug a build issue or memory error again though

it's definitely true in real life. i think there is actually a massive skill gap in using them effectively.

If the Code is bad, you can tell it to fix it to match your expectations —only at the edges, are you going to have to spell it out.

Lack of documentation used to be a running joke.

The goose_spec.txt in 2019 would be "goose is like c and rust only with syntax like array{;}[] btw it uses a small amount of stacks instead of malloc."0 10klocs.

Now, we make sure the AI leaves documentation so it can figure out what it was doing earlier.

A local model a few generations and many weights behind newest Claude can implement my novel algorithms and write the documentation I wouldn't.

Yes, many times it does dumb CS for kindergarten mistakes but it only has to work towards the optimum every once in a while.

Sadly there seems to be no point in arguing online though.

- You need to have a mental model to grow an idea

- When a pattern is known you can use a library!

- When a pattern is not known AI will make stuff up

No matter how "good" coding with an AI will become there is no way you get around learning, having someone understand what's going on and take responsibility.

I'm fine with formal checking and fuzzing or getting hints where a bug might come from - but these are iterations on tools we used before like a debugger, strict types, functional programming and well fuzzing.

I tried to have an AI iterate on a step i don't fully understand yet and i always regretted it. Highly interconnected projects will grind to a halt until you do learn.

You can check with people doing highly specced software in the medical domain that even with very good specs there is ambiguity in natural language thus it can never be the source of reliable and manageable software.

However - i read about some smart guys in the 40s or so encountering the need for a formal language to tell a computer what to do with 0 ambiguity and i think they called it a "programming language"

> I tried to have an AI iterate on a step i don't fully understand yet and i always regretted it.

I had to accept this one too even for pretty simple things. I had an LLM wire up a neovim connection handler for me without bothering to read the libraries docs or neovims RPC spec and I completely regretted it. You would thing such a small piece would be fine but I couldn't work on it and had to rip it out and rewrite it, and reading the AIs reference code actually got me more confused than anything.

I've found the sweet spot for me is just writing code and promoting an AI whenever I have a question or if I'm not sure the code will do what I think. When I hit a point and I groan about doing something, if it's small and I understand what needs to happen already I will ask the LLM I've been prompting already to do it. Usually by that point it's already gotten enough context from the back and forth on my questions or me explaining assumptions it got wrong when reviewing my code.

The tools have a learning curve. You can get extreme amounts of value out of them but you have to learn how to use them. Many tools are this way.

It might make sense to learn now, before everyone expects that you be an expert with this tool in order to do your job. On the other hand perhaps they will be easier to learn if you wait.

There is an incredible amount of nuance to be had in using LLMs and we reached the stage, probably only in the last few weeks where all the pieces have fallen into place such that if you are skilled at using the LLMs just about all of the previous problems with them can now be easily mitigated. If you learn how. There is a lot to learn.

how does one learn to mitigate not understanding the work they push , given that the work wasnt performed by them ?

By requiring the LLM to prove your requirements true mathematcially.

https://github.com/bendlang/bend

openai has retracted many of their lean proofs for the big drop of supposed novel mathematical solns. if frontier ai screws it up , how does this inconsistency manifest in weaker models?

"It might make sense to learn now, before ..." Since 2023, everything learned in Year X was 99% useless in year X+1.

"we reached the stage, probably only in the last few weeks where ..." This has also been said every other month since 2023.

> Since 2023, everything learned in Year X was 99% useless in year X+1.

Personally I disagree with this. I've been using LLMs the last few years and learning a lot about techniques for getting value out of them. Even though they weren't very good for a while they seemed worth learning to me because the rate of improvement was so fast. We've now reached a point where if you know how to use them right you can get lots of value out of them.

I really don't mean to be rude here but consider perhaps the reason why you're having trouble seeing why they're valuable is because you haven't used them enough in the last few years or kept up with the developments closely enough. Maybe I'm wrong. But consider it's possible you are too.

> This has also been said every other month since 2023.

I'm sure others have said that but my perspective has been that the models are not quite good enough yet, for a while but that the trajectory is so crazy that it made sense to make the bet and invest lots of time into learning how to use them on the assumption that they would soon be good.

Even now I would say they have a long way to go. What is hard but possible today will be easy tomorrow. But quite a lot is now possible with the right techniques. Crazy things.

I remember using LLMs a couple of years ago, back when there was no mention of "AI" in our country whatsoever. Not that much mention of it online either (much less widespread). I felt like I was holding a magic wand nobody else knew existed. :P

I wouldn't say that's true at all. My critical reasoning skills have improved immensely as I have learned how to find the flaws in the LLM's logic, and I am much better at code review in general than I was when I first started letting agents write code.

It really is like working with an over-eager employee. They just also happen to be a pathological liar.

Sounds like a skill issue.

This product is proof that you apparently either don't know how to use AI, haven't used it within the last 3 months, or don't know what you're talking about.

I just wrote a clean plugin in Rust. I don't know rust. It took work to avoid, remove, and replace slop, avoid spurious and pointless tests, etc. It wasn't "fire and forget". but it wrote the code, the code has been made clean, integrates well with existing project structure and does what it's supposed to. I spent a day or two on it, but had I attempted to write it from scratch, even knowing rust, it easily would have taken me longer.

Considering that I don't know rust at all, it's awesome.