It's also the type of thing where it's extremely unlikely to bullshit.

And ALSO the type of thing where I think you can work out if it's bullshitting you on your own; if it works, it's not bullshitting.

With technical things "bullshitting" is less of a problem than "wrong version" type of stuff, IME.

GDB... probably a slowly enough moving target, though. ;)

And even on the things I've seen it get wrong, even in higher-churn areas like front-end-Javascript, it's usually "usefully wrong" in that now I've tried one more thing and often end up aware of several more things to look up to try to figure out the right one.

That's why LLMs are best suited to programming. You know pretty quickly if it's lied to you.

I’m sorry, but these arguments are terrible. I see them boiling down to this:

“Use GPT to do this thing!”

“But GPT is notorious for confidently getting things wrong”

“That’s ok! Because you’ll know it’s wrong!”

Maybe I’m a little sore right now because I literally just finished trying to have GPT write code for me and it gave me code with all kinds of syntax errors and mistakes. Yes, I know not to trust it because the compiler flagged the problems, but what now? I go write the code myself I guess. Yay GPT?

What problem is GPT solving here then?

You can paste the compiler errors in and it’ll revise. But I think it helps to get a sense for what it can do well and what it can’t. It struggles with stuff that involves gnarly type level shenanigans, for example. But every time I’ve struggled with it I ended up realizing I was asking the wrong question. It’s not a magical device that solves all problems for you. It accelerates you when you know very well where you are and have the right idea of where you want to go. Try using it for boring refactoring tasks.

It’s also not bad with explaining code, so it can help with the “where you are” bit, too.

It gets it wrong maybe as often as I do on my first shot, but it does it much faster -- MUCH MUCH faster if any of the tech involved is something I'm not familiar with. And no matter who writes the code, me or the machine, I still quickly read it right after it's written.

Look, I've been doing this for 25 years, both as a career and continuously on the side for fun, and I am definitely an expert in a lot of things at this point. I'm allergic to saying crap like that about myself but feel it's necessary in this case, because it has transformed how I use a computer in a few short weeks.

My very first attempt with this stuff was actually a distinctly miserable experience and then something clicked. Wish I could articulate exactly what, but it's now one of the most joyous experience I've had in my career.

Exactly. I used to dread the 20+ times a day I would have to go to Stack Overflow and wade through a mountain of horse shit in the hopes of finding an answer. Now I can clearly describe the problem as a programmer would and ChatGPT frequently gets the solution on the first or second try.

I had one bad experience last week where I burned 20 minutes as ChatGPT gave me one invalid approach after another. Then I consulted the docs and learned that waht I wanted to do explicitly couldn't be done with that library. Tens steps forward and one step back. Totally worth it.

When the space of possible commands is large like GDB, it can get you the right incantation faster. It’s great for ramping up on a language or tool you are not fluent in; for example “how do you do X in C++”.

It’s not great IME for writing large chunks of code, though some seem to be having good enough results there.

GPT is excellent for when you already know that the thing you want to do exists but you can't remember the exact syntax or API method. What might take me 5-10 minutes of googling and reading documentation GPT can usually describe with a single statement of what I'm looking for.

My theory, it's like gambling, sometime's it a payout others it's bland, so it your waiting for the dopamine hit.

I have bad news to tell you about our ability to verify if a program matches a description of what it does on all inputs.

Regardless of whether you wrote it or an LLM did, yeah.