>I think most programmers who claim that LLMs aren't useful are reacting emotionally.

I don't think that's true. Most programmers I speak to have been keen to try it out and reap some benefits.

The almost universal experience has been that it works for trivial problems, starts injecting mistakes for harder problems and goes completely off the rails for anything really difficult.

> I don't think that's true. Most programmers I speak to have been keen to try it out and reap some benefits.

I’ve been seeing the complete opposite. So it’s out there.

> Most programming is trivial

That's a bold statement, and incorrect, in my opinion.

At a junior level software development can be about churning out trivial code in a previously defined box. I don't think its fair to call that 'most programming'.

Probably overloading of the term "programming" is the issue here. Most "software engineering" is non-programming work. Most programming is not actually typing code.

Most of the time, when I am typing code, the code I am producing is trivial, however.

Think of all the menial stuff you must perform regardless of experience level. E.g. you change the return type of a function and now you have to unpack the results slightly differently. Traditional automated tools fail at this. But if you show some examples to Cursor, it quickly catches on to the pattern and start autocompleting semi-automatically (semi because you still have to put the cursor to the right place but then you can tab, tab, tab…).

Don't misunderstand. I am not making an assertion that GenAI tools for development are useless.

I am just pointing out that the thread parent started his logical climb at a step one that is incorrect: 'Most programming is trivial'.

Given that they got it wrong on step one, how good do you thing step ten is?

From my perspective, writing out the requirements for an AI to produce the code I want is just as easy as writing it myself. There are some types of boilerplate code that I can see being useful to produce with an LLM, but I don't write them often enough to warrant actually setting up the workflow.

Even with the debugging example, if I just read what I wrote I'll find the bug because I understand the language. For more complex bugs, I'd have to feed the LLM a large fraction of my codebase and at that point we're exceeding the level of understanding these things can have.

I would be pretty happy to see an AI that can do effective code reviews, but until that point I probably won't bother.

It’s always harder to build a mental model of the code written by someone else. No matter what, if you trust an LLM on small things in the long run you’ll trust it for bigger things. And the most code the LLM writes, the harder it is to build this mental construct. In the end it’ll be « it worked on 90% of cases so we trust it ». And who will debug 300 millions of code written by a machine that no one read based on trust ?

It's reasonable to say that LLMs are not completely useless. There is also a very valid case to make that LLMs are not good at generating production ready code. I have found asking LLMs to make me Nix flakes to be a very nice way to make use of Nix without learning the Nix language.

As an example of not being production ready: I recently tried to use ChatGPT-4 to provide me with a script to manage my gmail labels. The APIs for these are all online, I didn't want to read them. ChatGPT-4 gave me a workable PoC that was extremely slow because it was using inefficient APIs. It then lied to me about better APIs existing and I realized that when reading the docs. The "vibes" outcome of this is that it can produce working slop code. For the curious I discuss this in more specific detail at: https://er4hn.info/blog/2024.10.26-gmail-labels/#using-ai-to...

I find a recurring theme in these kind of comments where people seem to blame their laziness on the tool. The problem is not that the tools are imperfect, it’s that you apparently use them in situations where you expect perfection.

Does a carpenter blame their hammer when it fails to drive in a screw?

I'd argue that a closer analogy is I bought a laser based measuring device. I point it a distant point and it tells me the distance from the tip of the device to that point. Many people are excited that this tool will replace rulers and measuring tapes because of the ease of use.

However this laser measuring tool is accurate within a range. There's a lot of factors that affect it's accuracy like time of day, how you hold it, the material you point it at, etc. Sometimes these accuracy errors are minimal, sometimes they are pretty big. You end up getting a lot of measurements that seem "close enough". but you still need to ask if each one is correct. "Measure Twice, Cut Once" begins to require one measurement with the laser tool and once with the conventional tool when accuracy matters.

One could have a convoluted analogy where the carpenter has an electric hammer that for some reason has a rounded head that does cause some number of nails to not go in cleanly, but I like my analogy better :)

>Does a carpenter blame their hammer when it fails to drive in a screw?

That's the exact problem. I have plenty of screwdrivers but there's so much pressure from people not in carpentry telling me to use this shiny new army Swiss knife contraption. Will it work? Probably, if I'm just screwing in a few screws. Would I readily abandon my set of precision built, magnetic tip, etc. Screwdriver set for it? Definitely not.

I'm sure it's great for non-carpenters to have so many tools in so small a space. But I developed skills and tools already. My job isn't just to screw in a few screws a day and call it quits. People wanting to replace me for a quarter the cost for this Swiss army carpenter will quickly see a quality difference and realize why it's not a solution to everything.

Or in the software sense, maybe they are fine with unlevel shelves and hanging nails in carpet. It's certainly not work I'd find acceptable.

They are useful, but so far, I haven't seen LLMs being obviously more useful than stackoverflow. It might generate code closer to what I need than what I find already coded, but it also produces buggier code. Sometimes it will show me a function I wasn't aware of or approach I wouldn't have considered, but I have to balance that with all the other attempts that didn't produce something useful.

> Lots of non-trivial programming tasks can be broken down into pure, trivial sections. Then, the non-trivial part becomes knowing how the entire system fits together.

I think that’s exactly right. I used to have to create the puzzle pieces and then fit them together. Now, a lot of the time something else makes the piece and I’m just doing the fitting together part. Whether there will come a day when we just need to describe the completed puzzle remains to be seen.

A good farmer isn’t likely to complain about getting a new tractor. But it might put a few horses out of work.

I would add that a lot of the time when I'm programming, I'm an expert on the problem domain but not the solution domain — that is, I know exactly what the pseudocode to solve my problem should look like; but I'm not necessarily fluent in the particular language and libraries/APIs I happen to have to use, in the particular codebase I'm working on, to operationalize that pseudocode.

LLMs are great at translating already-rigorously-thought-out pseudocode requirements, into a specific (non-esoteric) programming language, with calls to (popular) libraries/APIs of that language. They might make little mistakes — but so can human developers. If you're good at catching little mistakes, then this can still be faster!

For a concrete example of what I mean:

I hardly ever code in JavaScript; I'm mostly a backend developer. But sometimes I want to quickly fix a problem with our frontend that's preventing end-to-end testing; or I want to add a proof-of-concept frontend half to a new backend feature, to demonstrate to the frontend devs by example the way the frontend should be using the new API endpoint.

Now, I can sit down with a JS syntax + browser-DOM API cheat-sheet, and probably, eventually write correct code that doesn't accidentally e.g. incorrectly reject reject zero or empty strings because they're "false-y", or incorrectly interpolate the literal string "null" into a template string, or incorrectly try to call Element.setAttribute with a boolean true instead of an empty string (or any of JS's other thousand warts.) And I can do that because I have written some JS, and have been bitten by those things, just enough times now to recognize those JS code smells when I see them when reviewing code.

But just because I can recognize bad JS code, doesn't mean that I can instantly conjure to mind whole blocks of JS code that do everything right and avoid all those pitfalls. I know "the right way" exists, and I've probably even used it before, and I would know it if I saw it... but it's not "on the tip of my tongue" like it would be for languages I'm more familiar with. I'd probably need to look it up, or check-and-test in a REPL, or look at some other code in the codebase to verify how it's done.

With an LLM, though, I can just tell it the pseudocode (or equivalent code in a language I know better), get an initial attempt at the JS version of it out, immediately see whether it passes the "sniff test"; and if it doesn't, iterate just by pointing out my concerns in plain English — which will either result in code updated to solve the problem, or an explanation of why my concern isn't relevant. (Which, in the latter case, is a learning opportunity — but one to follow up in non-LLM sources.)

The product of this iteration process is basically the same JS code I would have written myself — the same code I wanted to write myself, but didn't remember exactly "how it went." But I didn't have to spend any time dredging my memory for "how it went." The LLM handled that part.

I would liken this to the difference between asking someone who knows anatomy but only ever does sculpture, to draw (rather than sculpt) someone's face; vs sitting the sculptor in front of a professional illustrator (who also knows anatomy), and having the sculptor describe the person's face to the illustrator in anatomical terms, with the sketch being iteratively improved through conversation and observation. The illustrator won't perfectly understand the requirements of the sculptor immediately — but the illustrator is still a lot more fluent in the medium than the sculptor is; and both parties have all the required knowledge of the domain (anatomy) to communicate efficiently about the sculptor's vision. So it still goes faster!

Yes. Productivity tools make programmer time more valuable, not less. This is basic economics. You’re now able to generate more value per hour than before.

(Or if you’re being paid to waste time, maybe consider coding in assembly?)

So don’t be afraid. Learn to use the tools. They’re not magic, so stop expecting that. It’s like anything else, good at some things and not others.

Trivial is fine but as you compound all the triviality the system starts to have a difficult time with putting it together. I don't expect it to nail it but then you have to unwind everything and figure out the issues so it isn't all gravy - fair bit of debug.

> people who can do it will be high status

They don't have high status even today, imagine in a world where they will be seen as just reviewers for AI code...

> They don't have high status even today

Try putting on a dating website that you work at Google vs you work in agriculture and tell us which yielded more dates.

Does it matter? I imagine the tanned shirtless farmer would get more hits than the pasty million dollar salary Googler anyway. (no offense to Googleers).

With so many hits, it's about hitting all the checkmarks instead of minmaxing on one check.

You can't just arbitrarily change (confounding) variables like that for a proper experiment. All other factors (including physique) must remain the same while you change one thing only: occupation.

"confounding" implies occupancy doesn't influce other factors of your life. I'm sure everyone wants the supermodel millionaire genius who's perfectly in touch with the feelings of their parter. If that was the norm then sure, farmers would be in trouble.

My comment was more a critique on online dating culture and the values it weighs compared to in person meetups.

I think it’s possible to create 2 dating profiles with the same pictures and change occupation only. It doesn’t have to be real to measure the impact of occupation.

> I think most programmers who claim that LLMs aren't useful are reacting emotionally. They don't want LLMs to be useful because, in their eyes, that would lower the status of programming.

I think revealing the domain each programmer works in and asking in hose domains would reveal obvious trends. I imagine if you work in Web that you'll get workable enough AI gen code, but something like High Performance computing would get slop worse than copying and lasting the first result on Stackoverflow.

A model is only as good as its learning set, and not all types are code are readily able to be indexable.