And junior devs are horrible at knowing what problem to solve and how to solve it without handholding. I am working on a relatively complex DevOps/“cloud application modernization” project. Where the heavy lifting is designing the process and gathering requirements. But there are a lot of 20-40 line Lambdas and Python/boto3 (AWS SDK), yaml/json wrangling, dynamic Cloudformation creating scripts.

I was able to give ChatGPT the requirements for all of them. The types of bugs I found during the first pass:

- the AWS SDK and the underlying API only returns 50 results in one call most of the time. From the SDK you have to use the built in “paginators”. ChatGPT didn’t use them the first time. But once I said “this will only return the first 50 results”. It immediately corrected the script and used the paginator. I have also had to look out for similar bugs from junior devs.

- The usual yaml library for Python doesn’t play nicely with CloudFormation templates because of the function syntax that starts with an “!”. I didn’t know this beforehand. But once I told ChatGPT the error, it replaced the yaml handling with cfn-flip.

- I couldn’t figure out for the life of me how to combine the !If function in CloudFormation with a Condition, and a Yaml block that contain another !Select function with two arguments. I put the template block without the conditional and told ChatGPT “make the VPC configuration optional based on a parameter”. It created the Parameter section, the condition and the appropriate Yaml.

I’ve given similar problems to interns/junior devs before and ChatGPT was much better at it.

Reading that I can only assume you’ve made really poor use of Jr developers.

Which ok I get why you think ChatGPT is more useful.

You really think that Jr devs could crank out the same code faster than ChatGPT? I couldn’t crank out the same code and you couldn’t either. The most you can hope from JR devs (even the ones I have met at BigTech) is that they don’t eat the chalk during the first 3-6 months.

As for now, issue with ChatGPT is that it doesn't really crank anything. It instantly produces answer for given input. While programmer can crank things. For example I asked ChatGPT to write a function which returns UUID generated with some rules. It spewed the solution. It looked like a correct one but when I run it, it returned wrong answer. I worked with ChatGPT for some time and it corrected its code. But I would expect from junior developer to actually run his code and check output.

Now if ChatGPT would be able to actually work on the problem rather than returning generated text, that would be a completely different beast. And I think that this workflow will come in the near future because it's pretty obvious idea. Get task specification, generate tests, generate code, fix code until tests work, refactor code until it meets some standards, etc.

> But I would expect from junior developer to actually run his code and check output.

You give junior devs way too much credit. They rarely test corner cases.

You can loop chatgpt around automatically, asking it to write tests and reason about the code for a few iterations; in my experience it auto corrects the code like a human would after some ‘thinking’ time. Of course the code has to run automatically and errors fed back, like with a human. It works fine though, without human input after some prompting work.

> I think that this workflow will come in the near future because it's pretty obvious idea. Get task specification, generate tests, generate code, fix code until tests work, refactor code until it meets some standards, etc.

ChatGPT probably works great if you use it to speedrun normal best practices in software engineering. Make it start by writing tests given a spec, then make it write code that will pass the specific tests it just wrote. I’m guessing it’ll avoid a lot of mistakes, much like any engineer, if you force it to do TDD.

Always hire a senior developer without experience for junior role. By that I mean hire a developer who knows how to program but lacks specific experience or has no formal experience at all.