Ask HN: How are you using LLMs or ML to improve your SWE?

Re:Ask HN: How are you using LLMs or ML to improve your SWE? by smoldesu on Sunday November 12, 2023 @04:54PM (#38244475)

It's great that you can take a different approach to achieve the same outcome. For one-off scripts with less context than answer, GPT will "work magic" as you've described.

Now, zoom out to what people actually work on. Typescript projects careening past 10k SLOC, microservice architectures with a mile-wide helm chart, and highly situational firmware programming that depends on proprietary datasheets. Unless you're an utterly disenfranchised CEO (see: Elon Musk, et. al), you will quickly realize that nobody gets paid to work on anything simple. Past a certain point, surface-level "copy and paste" answers stop being helpful. This is where I'd consider that a developer relying on ChatGPT becomes negligent; many projects force you to trade in your "let's try this snippet" instinct for investigative problem-solving. Hitting the reroll button on ChatGPT won't get you any closer to understanding the issue or preventing it from happening in the future.

I'm not entirely bearish on AI or LLMs, but if you can't see their limitations then I recommend pushing them to their limits. Ask ChatGPT something impossible (write an OS) and work your way down to more simpler requests. In my experience, every current AI model gives up long before writing a complete piece of software. Even working on smaller apps and programs (>500 lines) feels like a struggle, with current context limitations.

[ Parent | on HN ]

Re:Ask HN: How are you using LLMs or ML to improve your SWE? by catlover76 on Sunday November 12, 2023 @05:04PM (#38244565)

I think you may be misunderstanding how I and others are using it; I feel you are viewing it as very much an either-or proposition of "it either writes code for me or tells me exactly what's wrong" or it's unusable. For example, there are still questions one would have as one goes along for "investigative problem-solving" which GPT-4 is useful for answering.

> In my experience, every current AI model gives up long before writing a complete piece of software.

I don't get why you want to get it to write a complete piece of software.

[ Parent | on HN ]

Re:Ask HN: How are you using LLMs or ML to improve your SWE? by smoldesu on Sunday November 12, 2023 @05:22PM (#38244716)

Agree or disagree with it, my overall point is that once you leave the safe-harbor of simple scripting, GPT falls off hard. It's entirely possible that I've just been pushing it too hard, but for a lot of queries I get conflicting, wrong or outdated suggestions. Trying to shoehorn it in as a problem-solver in those situations has repeatedly failed on me.

> I feel you are viewing it as very much an either-or proposition of "it either writes code for me or tells me exactly what's wrong" or it's unusable.

Yeah, kinda. If ChatGPT spends it's output tokens telling you wrong, unnecessary or unhelpful information, it's not usable. Once it passes a certain bar of unreliability, it becomes useless. I'm not sure any place I've worked at in the past would feel comfortable paying me to roll those dice with ChatGPT on their systems.

I'm not against people empowering themselves by delegating AI for programming. Knock yourself out, if you want to develop like that it is entirely your prerogative.

[ Parent | on HN ]