News for hackers. Stuff that matters.

A comment on We need to check the gen AI hype and get back to reality.

Re:We need to check the gen AI hype and get back to reality by say_it_as_it_is on Thursday September 05, 2024 @01:56PM (#41458996)
> This study evaluates the impact of generative AI on software developer productivity by analyzing data from three randomized controlled trials conducted at Microsoft, Accenture, and an anonymous Fortune 100 electronics manufacturing company. These field experiments, which were run by the companies as part of their ordinary course of business, provided a randomly selected subset of developers with access to GitHub Copilot, an AI-based coding assistant that suggests intelligent code completions. Though each separate experiment is noisy, combined across all three experiments and 4,867 software developers, our analysis reveals a 26.08% increase (SE: 10.3%) in the number of completed tasks among developers using the AI tool. Notably, less experienced developers showed higher adoption rates and greater productivity gains.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4945566

[ Parent | on HN ]
Re:We need to check the gen AI hype and get back to reality by lumost on Thursday September 05, 2024 @02:05PM (#41459058)
A 26% productivity increase is... pretty big? There is obviously some noise in the data. But given copilots are pretty new pieces of tech, it does not seem impossible that there could be at least a 2x productivity growth available within the next 6-8 years. Given that SWE work is not particularly differentiated from CAD and similar types of knowledge work... is it really that unreasonable that AI could double the productivity of knowledge workers by 2030?

That's certainly worth a few trillion dollars in economic growth.

[ Parent | on HN ]
Re:We need to check the gen AI hype and get back to reality by svachalek on Thursday September 05, 2024 @02:06PM (#41459063)
The last statement rings very true for me. When I'm coding up something in my bread and butter language and libraries, current LLMs can rarely help me do something faster than I can do it myself. But when I'm trying to write a script in an unfamiliar language, it's much easier to let an LLM do it and fix the parts it got wrong, than to figure out how to write it all in the first place.
[ Parent | on HN ]