I find ChatGPT is pretty helpful for debugging. A big chunk of debugging is missing stupid things, and for debugging, it's like having a second set of eyes.

It's almost useless for high-level design and architecture. Those tasks do take more time than writing code, but they're super-charged by rapid prototyping, which ChatGPT excels at. For example, right now, if I'm picking between three libraries, I can have working versions of my code implemented in all three in a few minutes, and built out a bigger mockup in a few hours. I can make a more educated decision from there. In the olden days, that'd be a few hours and many days, respectively.

It also does really nicely for code reviews, which results in better code. That also speeds up maintenance and, later, debugging. That's especially true in domains I'm less familiar with.

Some maintenance, it excels at. I'm the author of a major piece of open source infrastructure, which is widely used, well-architected, but every technology used is obsolete (JQuery, older Python web frameworks, legacy database, etc.). I feel like I could rewrite the whole thing in React+Node+etc. in a few weeks. There is close to zero architecture work -- it's mostly having GPT write -- or sometimes just translate -- code piece-by-piece.

I don't feel like GPT-4 obsoletes software engineers as it's used yet, but it does really raise the bar on what's possible to build.

I’ve asked GPT to refactor blocks of code and it sometimes does a really good job, sometimes it makes it extremely verbose and wordy but you can’t always win.

I just hate how slow it is even with Plus. For analyzing code for debugging you’d ideally have something that moves a little faster.

I guess we’re in the 56k days of LLM watching it print line by line.

Not yet; I spend 60% my day reviewing PRs. I don't have to if chat GPT gets integrated as a PR reviewer..

ChatGPT will turn your day into spending 90% of your time reviewing code...

what do you mean? Do you mean that there is infinite amount of code because its also writing so much code?

Because people will generate more crap and throw it at you as the quality control. They'll have less knowledge overall as they cede more to chat GPT. Also your impact will be in cr, so they'll send you more people. You get faster and more responsive using chat GPT to review, so you'll do more reviews, causing people to send more frequent ones to get your feedback faster. Eventually just hire some interns to toss plausible shit at gpt and then you. Fire the ftes. Done.

That's actually how I spent a lot of my time. GPT drafts code of mixed quality, and I spend a lot of time reviewing it.

What's interesting is that GPT isn't trained to produce high-quality code. It's trained as an autocompletion tool. I'd be curious how smart such a system could be if we knew how to engineer it for quality.

I review PRs primarily for correctness and to look for nonobvious problems. I rarely find buggy code. Sometimes I'll find performance issues. In order to do what I do when reviewing, ChatGPT would need clear specs (yeah right) and a deep understanding of our system, domain, and even our customers.

Of course I'll try it and see how it does. But it struggled and failed to produce what I would consider trivial code when I tried to create a single usable page to spec in react native, so I'm not terribly optimistic.

GPT can't do full code reviews. GPT is very good at finding improvements, such as making code more functional, bad variable names, etc. In some sense, it's like ESLint or pycodestyle on steroids.

It won't find subtle race conditions, abstraction violations, or deeper issues.

I find it makes code reviews shorter and more productive, so perhaps you'll spend 10% of your time instead of 60%. You won't spend 0% of your time, though.

Especially that code generators are pretty old, ChatGPT is very late to the game. I see all this people spending time to create a landing page with ChatGPT whereas in reality you can just install Wordpress and it even looks nicer.

As though creating landing pages is where it's tricky. So many cookie cutter examples to just create one.

One of the examples from GPT wasn't even responsive and didn't work on mobile.

But...some folks are creating iOS games without any Swift experience, so that's cool.

Which is why I still believe proper developers will still have jobs :)

GPT is just Spreadsheets in a way. Lot of companies had business users and SMEs create contraptions based on Excel and then when it's a hairy mess or a rube goldberg machine, there are "digital transformation" projects that are done to make them proper apps.

That might be the exact right analogy. People who see Excel for the first time seem to think they have superpowers. In actual fact the superpower is knowing how to use it effectively and knowing when you are on the edge of its abilities.

Plain and simple: Feed the code it produced back into it and ask it to modify the sections needing maintenance or changes, or just tell it the issues you're having and ask it to explain/debug them.

IMO, the problem is that it doesn't work. It could introduce other bugs, make incorrect changes, or mess up the code. If a good developer does this, it converges, with GPT I am not so sure.

We need to find a way for the system to iterate, debug, write unit tests, and repeat the loop. In theory it's doable, but we are yet to see the system which does this.

Be careful! I have been experimenting with just this approach and ChatGPT 4 often forgets important parts of the code when generating the new files. If you ask ChatGPT to break down every change into step-by-step changes with code examples, it does a really great job and you can check every change yourself before implementing it.

A good prompt is key. If pasting code make sure to run it through the tokenizer first to make sure it’s not too long. Be very descriptive about what you want it to do. If it goes off the rails just hit the stop button and then edit your prompt with the little edit button at the top right and try again, this saves on context length and it saves the previous prompt and response so you can switch between them and compare.

GPT is a much better debugger than coder in my experience. It is a drop in replacement for googling random linux errors too.

"I'm getting this error <error> with the following code. Please add log statements that will test for all the things that could be going wrong <code>"

Saved me about 15 minutes yesterday. Not only does chatGPT generate the probably causes of the error, it knows how to test the code for those errors. The human doesn't even have to read the error message anymore.