News for hackers. Stuff that matters.

A comment on Claude 3 beats GPT-4 on Aider's code editing benchmark. Parent comment

Re:Claude 3 beats GPT-4 on Aider's code editing benchmark by varispeed on Wednesday March 27, 2024 @10:06AM (#39839293)
It is still remarkably better than working with junior developers. I can ask it to code something based on specification and most of the time it will do better job than if I send the same task to a human.

It often makes some small mistakes, but little nudge here and there corrects them, whereas with human you have to spend a lot of time explaining why something is wrong and why this and that way would rectify it.

The difference is probably because GPT has access to a vast array of knowledge a human can't reasonably compete with.

[ Parent | on HN ]
Re:Claude 3 beats GPT-4 on Aider's code editing benchmark by throwaway35777 on Wednesday March 27, 2024 @10:42AM (#39839861)
> It is still remarkably better than working with junior developers

It really bugs me when people imply AI will replace only certain devs based on their title.

Seniority does not equal skill. There are plenty of extremely talented "junior" developers who don't have the senior title because of a slow promo process and minimum time in role requirements. They can and do own entire projects and take on senior-level responsibilities.

I've also worked with a "senior" dev who struggled for over a month to make a logging change.

[ Parent | on HN ]
Re:Claude 3 beats GPT-4 on Aider's code editing benchmark by bigcoke on Wednesday March 27, 2024 @10:53AM (#39840052)
that's more or less how I use gpt-4, "how to <describe some algorithm in some language><maybe some useful context with a short code example>", most of time it output something useful that I can work with, but if a junior work is to be my gpt-4, that's a waste of human resource.
[ Parent | on HN ]
Re:Claude 3 beats GPT-4 on Aider's code editing benchmark by aiisjustanif on Wednesday March 27, 2024 @10:25AM (#39839573)
> It is still remarkably better than working with junior developers.

Definitely not all junior developers. I have yet to see it do well at handling code migrations, updating APIs, writing end to end tests, and front end code changes with existing UX specifications to name a few things.

[ Parent | on HN ]
Re:Claude 3 beats GPT-4 on Aider's code editing benchmark by greenavocado on Wednesday March 27, 2024 @10:40AM (#39839821)
Wow, you have good junior developers.
[ Parent | on HN ]
Re:Claude 3 beats GPT-4 on Aider's code editing benchmark by throwaway35777 on Wednesday March 27, 2024 @10:50AM (#39839986)
Work at a good company and you'll find that the juniors are pretty capable.
[ Parent | on HN ]