News for hackers. Stuff that matters.

A comment on ChatGPT Passes Google Coding Interview for Level 3 Engineer with $183K Salary. Parent comment

Re:ChatGPT Passes Google Coding Interview for Level 3 Engine... by vbezhenar on Saturday February 04, 2023 @11:59PM (#34662061)
As for now, issue with ChatGPT is that it doesn't really crank anything. It instantly produces answer for given input. While programmer can crank things. For example I asked ChatGPT to write a function which returns UUID generated with some rules. It spewed the solution. It looked like a correct one but when I run it, it returned wrong answer. I worked with ChatGPT for some time and it corrected its code. But I would expect from junior developer to actually run his code and check output.

Now if ChatGPT would be able to actually work on the problem rather than returning generated text, that would be a completely different beast. And I think that this workflow will come in the near future because it's pretty obvious idea. Get task specification, generate tests, generate code, fix code until tests work, refactor code until it meets some standards, etc.

[ Parent | on HN ]
Re:ChatGPT Passes Google Coding Interview for Level 3 Engine... by scarface74 on Sunday February 05, 2023 @01:03AM (#34662392)
> But I would expect from junior developer to actually run his code and check output.

You give junior devs way too much credit. They rarely test corner cases.

[ Parent | on HN ]
Re:ChatGPT Passes Google Coding Interview for Level 3 Engine... by anonzzzies on Sunday February 05, 2023 @01:06AM (#34662402)
You can loop chatgpt around automatically, asking it to write tests and reason about the code for a few iterations; in my experience it auto corrects the code like a human would after some ‘thinking’ time. Of course the code has to run automatically and errors fed back, like with a human. It works fine though, without human input after some prompting work.
[ Parent | on HN ]
Re:ChatGPT Passes Google Coding Interview for Level 3 Engine... by mullingitover on Sunday February 05, 2023 @01:22AM (#34662484)
> I think that this workflow will come in the near future because it's pretty obvious idea. Get task specification, generate tests, generate code, fix code until tests work, refactor code until it meets some standards, etc.

ChatGPT probably works great if you use it to speedrun normal best practices in software engineering. Make it start by writing tests given a spec, then make it write code that will pass the specific tests it just wrote. I’m guessing it’ll avoid a lot of mistakes, much like any engineer, if you force it to do TDD.

[ Parent | on HN ]