News for hackers. Stuff that matters.

A comment on Effects of Gen AI on High Skilled Work: Experiments with Software Developers. Parent comment

Re:Effects of Gen AI on High Skilled Work: Experiments with ... by nerdjon on Friday September 06, 2024 @09:07AM (#41465827)
I am curious about this also. I have now seen multiple PR's that I had to review that a method was clearly completely modified by AI with no good reason and when asked why something was changed we just got silence. Not exaggerating, literal silence and then trying to ignore the question and explain the thing we were first asking them to do. Clearly having no idea what is actually in this PR.

This was done because we asked for a minor change to be done (talking maybe 5 lines of code) and tested. So now not only are we dealing with new debt, we are dealing with code that no one can explain why it was completely changed (and some of the changes were changes for the sake of change), and we are dealing with those of us that manage this code now looking at completely foreign code.

I keep seeing this with people that are using these tools and they are not higher level engineers. We finally got to the point of denying these PR's and saying to go back and do it again. Loosing any of the time that was theoretically gained from doing it in the first place.

Not saying these tools don't have a place. But people are using it without understanding what it is putting out and not understanding the long term effects it will have on a code base.

[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by okwhateverdude on Friday September 06, 2024 @02:51PM (#41468741)
> Not saying these tools don't have a place. But people are using it without understanding what it is putting out and not understanding the long term effects it will have on a code base.

It is worse than that. We're all maintaining in our heads the mental sand castle that is the system the code base represents. The abuse of the autocoder erodes that sand castle because the intentions of the changes, which are crucial for mentally updating the sand castle, are not communicated (because they are unknowable). This is same thing with poor commit messages, or poor documentation around requirements/business processes. With enough erosion, plus expected turn over in staff, the sand castle is actually gone.

[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by rongenre on Friday September 06, 2024 @04:46PM (#41469585)
There needs to be a pattern for including AI-generated code, recording all the context which led to the generation.
[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by jcgrillo on Friday September 06, 2024 @06:32PM (#41470184)
Easy: exclude developers who try it. Learning--be it a new codebase, a new programming language, a new database--takes time. You're not going to be as productive until you learn how. That's fine! Cheating on your homework with an LLM should not be something we celebrate, though, because the learner will never become productive that way, and they won't understand the code they're submitting for review.
[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by XenophileJKO on Saturday September 07, 2024 @03:15AM (#41472114)
The truth is that the tools are actually quite good already. If you know what you are doing they will 10-20x your productivity.

Ultimately not adopting them will religate you to the same fate as assembly programmers. Sure there are place for it, but you won't be able to get near as much functionally done in the same amount of time and there won't be as much demand for it.

[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by shinycode on Saturday September 07, 2024 @06:37AM (#41473021)
Do you agree that the brain-memory activity of writing code and reading someone else’s code is totally different ? The sand castle analogy is still valid here because once you have a x10 productivity or worse a x20 one, there is no way you can deeply understand the things the same way than if you wrote it from scratch. Without spending a considerable amount of time and getting productivity down, the understanding is not the same. If no one is responsible because it’s crap software and you won’t be around enough time to bear responsibility… it’s ok I guess ?
[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by andai on Saturday September 07, 2024 @06:39AM (#41473037)
I write at 1x speed and I can't even understand my own code the next day.
[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by kragen on Saturday September 07, 2024 @07:08AM (#41473184)
if you are seeing 900% productivity gains, why did these controlled experiments only find 28%, mostly among programmers who don't know what they're doing? do you have any hypotheses?
[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by dasil003 on Saturday September 07, 2024 @09:01AM (#41473703)
I think you’re probably right, though I fear what it will mean for software quality. The transition from assembly to high level languages was about making it easier to both write and to understand code. AI really just accelerates writing with no advancement in legibility.
[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by jcgrillo on Saturday September 07, 2024 @10:14AM (#41474054)
We'll see. Get back to me sometime in 2034 and we can compare notes.
[ Parent | on HN ]
Re:Effects of Gen AI on High Skilled Work: Experiments with ... by jonnycomputer on Saturday September 07, 2024 @09:44AM (#41473909)
When using code assist, I've occasionally found some perplexing changes to my code I didn't remember making (and wouldn't have made). Can be pretty frustrating.
[ Parent | on HN ]