News for hackers. Stuff that matters.
Re:The Leverage of LLMs for Individuals by the_doctah on Wednesday May 10, 2023 @11:25AM (#35888758)
People who use ChatGPT to spit out code as a method of learning software engineering are going to be seriously deficient in fundamentals like performance.

I've already worked with way too many engineers who copy/paste code without understanding what it does.

Re:The Leverage of LLMs for Individuals by stasmo on Wednesday May 10, 2023 @11:35AM (#35888902)
The good news is that those copy/pasters are probably going to be copy/pasting higher quality code if they’re using GPT4.
Re:The Leverage of LLMs for Individuals by hoorayimhelping on Wednesday May 10, 2023 @11:57AM (#35889229)
I don't think this is good news, it's at best not-bad news. We're already inundated with content, most of it copycatting other content, and it's replacing meaningful things, like archetypal stories and philosophy. Look at how people describe Marvel movies and Star Wars shows - it's content, not stories. Scaling the ability to just repeat things without an understanding of those things will at best just add more content. It will likely contribute to the further creation of complex systems people don't understand and can't control.

"Um, I'll tell you the problem with the scientific power that you're, that you're using here. It didn't require any discipline to attain it. Ya know, you read what others had done, and you, and you took took the next step. You didn't earn the knowledge for yourselves, so you don't take any responsibility for it. You stood on the shoulders of geniuses, uh, to accomplish something as fast as you could..." (Emphasis mine).

https://www.youtube.com/watch?v=4PLvdmifDSk&t=61s

Re:The Leverage of LLMs for Individuals by fleetwoodsnack on Wednesday May 10, 2023 @01:08PM (#35890246)
Just about any original scientific text or undertaking begins with a literature review.
Re:The Leverage of LLMs for Individuals by nivenkos on Wednesday May 10, 2023 @11:46AM (#35889052)
But the awesome part is that you can ask it to explain parts and it usually does pretty well.

I was using it for debugging WINE - where I have no DirectDraw4 / DirectX6 experience, and it's much nicer than trying to trawl through ancient MS documentation myself.

Re:The Leverage of LLMs for Individuals by simonw on Wednesday May 10, 2023 @12:01PM (#35889289)
I think this is currently an open question.

It may turn out that ChatGPT results in a new generation of developers who produce terrible code, full of flaws, with no understanding of how it actually works.

Or... it might be that ChatGPT helps newcomers get over that horrendous initial learning curve, start by writing bad code... and then get better at it.

I'm currently cautiously optimistic that the latter scenario is going to win out.

Re:The Leverage of LLMs for Individuals by protosevn on Wednesday May 10, 2023 @12:26PM (#35889638)
I'm on year two of my journey to become a game developer, i am focusing on programming and if i realize i don´t want to work with games in the future at least i have solid knowledge on how to deal with a broad area of software problems.

I don´t rely on ChatGPT to write any code for me but i do use it whenever there´s a concept i don´t understand and if there´s any problem blocking me from getting forward in my many programming books. Having a tireless teacher/mentor with extensive knowledge and who does not care however stupid question you ask is a very good tool for learning.

Re:The Leverage of LLMs for Individuals by patapong on Wednesday May 10, 2023 @12:13PM (#35889468)
I share your optimism.

ChatGPT could also empower individuals who may not have considered learning programming to create practical programs that enhance their daily routines, similar to how VB6 significantly reduced the barrier to entry for developing various applications.

Re:The Leverage of LLMs for Individuals by sha016 on Wednesday May 10, 2023 @01:01PM (#35890136)
I don't think it will help people learn as much as having to write everything yourself. Sure it will get them productive 10x faster, as a shortcut. You can only get so good at riding a bicycle by watching others ride a bike. Remember that caching issue that cost you hours of debugging? You don't forget that. Maybe the generated code doesn't produce a bug this time, but the 5th or 6th time? It's hard enough to debug code you wrote yourself sometimes, let alone generated code.
Re:The Leverage of LLMs for Individuals by ska on Wednesday May 10, 2023 @01:11PM (#35890293)
In my experience newcomers and junior engineers don't get better by writing more bad code. They learn mostly by 3 methods. 1) finding flaws in their code the hard way (slow) 2) getting careful feedback from more experienced people (has to be done well) and 3) reading better code for a problem they already understand.

I'm not sure GPT helps with any of those yet, as it seems to be more likely to spit out incorrect-to-sortof-okayish so far.

Re:The Leverage of LLMs for Individuals by pulvinar on Wednesday May 10, 2023 @01:25PM (#35890482)
It goes the other way too. I've had it generate code using better methods or features new to me, when I thought I knew the language well.
Re:The Leverage of LLMs for Individuals by intelVISA on Wednesday May 10, 2023 @02:07PM (#35891065)
I'm the hallucinating LLM? And Other Such Crises it has given me.
Re:The Leverage of LLMs for Individuals by giobox on Wednesday May 10, 2023 @12:39PM (#35889819)
I've seen similar results with teams I work with. The strongest engineers who can spot the deficiencies or errors in LLM generated code generally get a huge productivity boost, as they can compare the "shape" of the output to what they would expect for a given problem.

The junior devs, lacking the experience that provides the context or knowledge to analyze the "shape", let all manner of minor and major issues from the generated code slip past.

I really notice this particularly with say less popular frameworks or code libraries, where many LLM code generators appear much more likely to hallucinate invalid code. I see similar issues with DSLs too, where the LLM starts to just guess at the DSL syntax. I really hate when an engineer new to a framework just goes to the LLM first, without at least spending a little time learning the framework for the task.

Re:The Leverage of LLMs for Individuals by stevenhuang on Wednesday May 10, 2023 @01:02PM (#35890148)
If performance is a concern, just re-prompt GPT if there are any performance concerns, and to propose fixes for them if there are.

This is called reflexion and has been found to be an effective way to improve response accuracy and usefulness (also reduces hallucinations).

https://arxiv.org/abs/2303.11366