In the same way I can imagine that by 2030 LLMs will still have memory problems and hallucinations. Although I’m sure by then we’ll have something better than pure LLMs.
I would pay for a pre-packaged system which could _locally_ and _privately_ make sense of all the emails, PDFs, slack messages, web pages I saw, other documents shared with me, code, etc etc, and make it all easily queryable with natural language, with references back to the sources. Sooner or later someone will make something like that.
[1] Technically LLMs do have a forget that last toke and go back so I can try again operation so this is only 99% true.
People have been committing terrible code to projects for decades now, long before AI.
The solution is a code review process that works, and accountability if experienced employees are approving commits without properly reviewing them.
AI shouldn't have anything to do with it. Bad code shouldn't be passing review period, no matter if it was AI-assisted or not. And if your org doesn't do code review, then that's the actual problem.
AI makes it much easier to push out bad code, fast...in a "frenzied" way one could say.
You’re putting the entire responsibility on senior employees. So we need much more of them. In fact, we don’t need juniors, because we can generate all possible code combinations. After all, it’s the responsibility of the seniors to select which one is correct.
It’s like how hiring was made crap by the “One-click apply” on LinkedIn and all other platforms. Sure it’s easy for thousands of people to apply. Fact is, we offer quite a good job with high salary, and were looking for 5 people. We’ve spent a full year selecting them, because we’ve receive hundreds of irrelevant applications, probably some AI-generated.
It’s no use to flood a filter with crap, hoping that the filter will do better work because it has a lot of input.
The answer is also the same.
Volume. AI makes it trivially easy to generate vast amounts of it that don’t betray their lack of coherence easily. As with much AI content, it creates arbitrary amounts of work for humans to have to sift through in order to know it’s right. And it gives confidence to those who don’t know very much to then start polluting the informationsphere with endless amounts of codswallop.
I wouldn't be comfortable with an accountant who couldn't do practical arithmetic in their head, or a surveyor who didn't have a fluent grasp of trigonometry and the ratios or function values for common angles.
Of course -- I don't care about the people working for them as a bookkeeper or assistent. They can be button monkeys as much as their boss lets them, but I also wouldn't expect those folk to reach very high in their career. Not everyone's going to, and a disinterest in and lack of technical fluency is a darn good predictor.
Your teacher was giving good advice about building skills and internalizing knowledge, because those are what contribute to mastery of a craft; maybe you were just being too pedantic or cynical to hear it?
The core idea is no, calculating in your head doesn’t turn you into a pro. Being a pro makes calculating in your head safe enough to rely on until you get to your desk and double-check.
That said, I was an integrator half my life and seen some accountants, big and small. Everyone used a calculator, it’s right next to their mouse and there’s another one elsewhere. And no one ever tried to talk about numbers afk, except for ballparks. Most of the times they suggest to go and query a db together. I find it very professional.
I have no interest interacting with someone who is going to get into how great the slide ruler is and how kids these days need to learn the slide ruler. When I was younger I thought this type of person was highly admirable but now older and wiser I see how full of shit they are.
> Does it really matter though? It sounds awfully like when a school teacher said you're not going to have an calculator in your pocket.
Yes. A younger me had teachers say that to me, and that younger me thought they were wrong.
But it turns out they were right, and younger me was wrong. Calculator dependence created a ceiling built from "educational debt" that many years later severely limited my mathematical ability.
The problem is focusing too much on getting the answer, and loosing sight of building the abilities that allow you to get there with your own mind. Eventually the need to manipulate your automated crutch turns into a bottleneck.
I wonder if that would be an issue if nobody had any.
Can you explain how your problems with arithmetic affected your ability to reason and prove things? I struggle to see how the two are connected.
It didn't affect my ability to reason and prove things, just as long those things don't strongly require the knowledge and skills I should have gotten from the calculator shaped gap in my education. I lack a lot of the background knowledge and/or familiarity and comfort with many skills that I should have.
Using AI to “help” learn to program is replacing the person’s effort thinking through the problems fully and is ultimately going to stunt their learning.
Forbidding calculators and requiring students to do mental math for absolutely everything is unnecessary. But requiring students to solve integrals by hand when they're learning about integrals? Entirely reasonable.
If your goal when teaching coding is to teach the mechanical process of writing code: sure, go ahead and use LLMs for that process. But if your goal is to develop a deeper understanding of how to code, then LLMs can very easily obscure that. The end goal is not always just the answer.
I remember having to write code on a piece of paper in university.
Curiously, this lead to a lot of other students not really learning or caring about indentations and attempting to make code formatted in a generally readable way once they actually got access to IDEs and such.
They'd write it much like you would regular text on a piece of paper and would get absolutely stumped with unbalanced brackets whenever some nested structure would be needed.
Not only that, but somehow a lot of them didn't quite get around to using the various refactoring features of the tools, since they treated them like editors for writing text.
Did it help me memorize the basic constructs of the languages in question? Absolutely not, at least in the long term, since nowadays I have to jump between a lot of different languages and none of the syntax quite sticks, rather a more high level approach to problem solving.
LLMs just make that easier for me, letting me offload more work around language constructs and boilerplate, albeit requiring occasional intervention.
Maybe my memory just sucks.
imagine if the calculator in your pocket gave you subtly wrong answers most of the time
The problem with AI-assisted coding is that, applied uncritically, it circumvents this understanding. And without it, you won't even be able to judge whether the code that CoPilot etc spit out is correct for the task.
For me personally, it’s a fantastic way to learn. I could definitely see people just using it and not actually learning, but to each their own.
Would it be better code if someone with 3 years of university and 5 years of coding practice did it? Yes, very probably, but the gap seems to be narrowing. Humorously I don't know enough about good code to tell you whether what I build with llms is good code. Sometime I build a function that feels magical, other times it seems like a fragile mess. But I don't know.
Do I know "javascript" or "python" or the theory of good coding practice? No, not currently. But I am building things. Things that I have personal, very specific requirements for. Where I don't have to liaise or berate someone else. Where I don't have to pay someone else. Where I don't share the recognition (if there is any ever) of the thing, I and only I have produced- (with chatGPT, Gemini and most recently llama3).
Folks have been feeling superior for 70 years and earning a good living because they spoke the intermediary language of compute engines. What makes them actually special NOW is computer science, the theory- the languages, we have very cheap (and in the case of open source local models free) translators for those now. And they can teach you some computer science as well, but that is still time and practice.
I'm the muggle. The blunt. And I'm loving this glowy lantern, this psi-rig.
When it comes to production quality code that should capture complex and/or business-critical functionality, you do want an experienced person to have architected the solution and to have written the code and for that code to have been tested and reviewed.
The risk right now is of many IT companies trying to win bids by throwing inexperienced devs at complex problems and committing to lower prices and timelines by procuring a USD 20 per month Github Co-Pilot subscription.
You individually may enjoy being able to put together solutions as a non-programmer. Good for you. I myself recently used ChatGPT to understand how to write a web-app using Rust and I was able to get things working with some trial and error so I understand your feeling of liberation and of accomplishment.
Many of us on this discussion thread work in teams and on projects where the code is written for professional reasons and for business outcomes. The discussion is therefore focused on the reliability of and the readability of AI-assisted coding.
What I really need is things like "show me the complete chain of code for this particular user activity in the app and highlight tokens used in authentication" ... - something senior engineers struggle to pull from our hundreds of services and huge pile of code. And so far sourcegraph and lightstep are incapable of doing that job. Maybe with better RAG or infinite context length or some other improvement there will be that tool. But currently the combined output of 1000's of engineers over years almost un-navigable. Some of that code might be crisp, some of it is definitely of llm-like quality (in a bad way)- I know this because I hear people's explanation of said code and how they misremembered it's function during post mortems. Folks copy and pasting outdated example code from the wiki etc. ie making things they don't understand. I presume that used to happen from stackoverflow too. Engineers moving to llm won't make too much difference IMO.
I agree, your points are valid, but I see "prompt engineering" as democratization of the ability to code. Previously this was all out of reach for me, behind a wall of memorization of language and syntax that I touched in the Pascal era and never crossed. 12 hours to build my first node.js app that did something in exactly the way I had wanted for 30 years. (including installing git and vscode on windows- see, now I am truly one to be reviled)
Copilot may work for simple scripts, but even for that where it mostly for things right, in my experience it still introduced subtle bugs and incorrect results more often than not.
Language models can copy the top answers from SO, ingest docs and specs etc. And then the information is never updated? Or are they going to train it from scratch? On what? Outdated github saved games?
You just need to ask it what to ask. /s
Isnt this what code reviews are for? I catch a decent amount of code that looks AI generated. Typically, some very foreign pattern or syntax that this engineers never used nor is common in the codebase. Or something weirdly obtuse that could be refactored and shows a lack of understanding.
Normally I ask something like, "Interesting approach! Is there a reason to do it this way over (mention a similar pattern in our codebase)?" or if it's egregious, I might ask, "Can you explain this to me?".
This feel similar to early career engineers copy pasting stack overflow code. Now its just faster and easier for them to do. It's still fairly easy to spot though.
Wouldn't you need to have people with a proper understanding of the programming language and framework to do the code reviews?
Listening to people talk in Japanese, I picked up enough to have some idea what they're talking about. I can't speak the language beyond some canned phrases. I certainly wouldn't claim to know Japanese. And I definitely wouldn't get a job writing books in Japanese, armed with Google Translate.
For programming, my solution to this right now is lots of pair programming. Really gives you a good idea of where somebody is at, what they still need to learn, and lots of teaching opportunities. I just hired a junior and we spend about 8 hours a week pairing.
Code reviews, done by existing more senior members of the project/team, should prevent this. If it isnt then the project has more issues than AI generated code.
Or maybe it's far more common than you realise, and you're only spotting the obvious ones.
The point being I think it depends on the project, it's size, audience, lifetime, etc whether or not code review is a win. my gut says it usually is a win but not always
At the one company I was where we did code reviews, they took at most one hour per day. The reviewer would go through the changes, ask for explanations when they didn't understand something, suggest small modifications to fit the codebase's style/structure/vibe, and move on.
They definitely didn't take 50% of our time. Half my time wasn't even spent coding, it was spent thinking about problems (and it was a workplace with an unusually high coding-over-thinking ratio, because of its good practices).
Imo introducing code reviews might very well decrease the velocity by 50% even if the actual time spent on code reviews is much less.
I see at least two reasons for this: 1) increased need for synchronization/communication 2) increased subjective frictions and mental overhead (if only for task switching).
Code reviews are not special in this respect, similar effects can be caused by any changes to the process.
Less time may be required if you have good rapport and long work relationship with the person, but I would say halving productivity sounds about right if we are talking mandatory reviews.
Using an LLM to generate code means having to do the same. The longer the autocompleted text, the more careful you have to be vetting it. Lower predictability, compared to a person and especially one you know well, means that time requirement would not go down too much.
Agree 100%, even when LLMs aren’t involved.
Certainly not all code reviews are like this—in many cases the approach is fundamentally sound, but you might have some suggestions to improve clarity or performance.
But where I work it’s not all that rare to open up a PR and find that the person has gone full speed in the wrong direction. In those cases you’re losing at least a morning to do a lot of the same leg work they did to find a solution for the problem, to then get them set off again down a better path.
Now in the ideal case they understood the problem they needed to solve and perhaps just didn’t have the knowledge or experience to know why their initial solution wasn’t good; in such a scenario the second attempt is usually much quicker and much higher quality. But introduce LLMs into the mix and you can make no guarantees about what, if anything, was understood. That’s where things really start to go off the rails imo.
Sure, but I mean you still have to determine whether it is sound or not in the first place; i.e., you must know how to solve the problem. You can only evaluate it as right or not if you have something to compare against, after all.
If you have worked with a person for a while, though, I can see that you can spot various patterns (and ultimately whether the approach is sane) faster thanks to established rapport. Not so with LLMs, as you probably agree.
It really doesn't need to be a massive amount of task-switching. And the benefits were obvious.
It probably also depends on the people that you're working with. I can easily imagine the velocity plummeting when the person reviewing your code loves to nitpick and bicker and ask for endless further changes with every next round of alterations that you do.
Doubly so if they want to do synchronous calls to go over everything and essentially create visible work out of code that already worked and was good enough in the first place.
I'm not saying that there aren't times like that for everyone occasionally, but there are people for whom that behaviour is a consistent pattern.
"One and done" products like games where you're mostly fixing bugs and releasing new content without major overhauls to the underlying code probably only need code reviews for code that is complex/clever where bugs are likely to be hiding, beyond that QA is fine.
reviews don't have to be formal to be reviews
I would have preferred to have someone to collaborate with at each of those places.
There is no substitute to doing something correctly in the first place. The problem is that in the real world, deadlines and lack of time will always cause the default solution to be accepted a small percentage of time even when it is not ideal. The increasing creep of AI will only exacerbate that and most technophiles will default into thinking of a new and improved AI tool to help with the problem, until it will be AI tools all the way down.
No thanks.
If you have an engineering culture that doesn't emphasize thorough code review (at least of juniors, leads and architects emergency-pushing is a different story) that's a problem. In addition to catching bugs, that's a major vector for passing on knowledge.
I tell you, it's spiralling out of control. Besides, even if you're doing the hiring, you may not have control if there is a profit motive. Us technical people cannot control the race to the bottom line, especially over the period of decades.
A foundational concept of quality control is to not rely on inspection to catch production defects. Why not? It diffuses responsibility, lets more problems get to the customer and is less efficient than doing it correctly to start with.
Code review is the feedback mechanism that allows quality control to improve the process, teaching the engineers how to do it right in the first place.
This is a widespread problem regardless of AI. Hence the myriad Stack Overflow users who are frustrated after asking insane questions and getting pushback, who then dig their heels in after being told the entire approach they’re using to solve a problem is bonkers and they’re going to run into endless problems continuing down the path their on.
Not that people aren’t on too fine a hair trigger for that kind of response. But the sensitivity of that reaction is a learned defense mechanism for the sheer volume of it.
The problem is, SO can't tell someone who asks an insane question from someone who asks the same question but has constraints that make it sane. *
So in time, sane people during unusual stuff stop asking questions and you're left with homework.
* For example, "we can't afford to refactor the whole codebase because some architecture astronaut on SO says so" is a constraint.
Or another nice one is "this is not and will never be a project that will handle google-like volumes of data".
Stack Overflow is not there to help you solve your use-case. It's there to create a body of knowledge that everyone can refer to. You need to spell out your specific reasons so that the question and answers become useful to others.
A lot of the friction on Stack Overflow comes from people thinking it's a free help website rather than an attempt to create a collaborative knowledgebase.
I was dumb and was trying to help people, not build a 'knowledge base'.
It is about helping people - but helping multiple people for the same effort it takes to help one person.
As we descend further into "Need help? Just ask on our Discord!" hell - it's a lesson that I wish resonated with more people.
It was nice to use early in its life when every answer was fresh. These days, the stale knowledge outweighs the fresh and the top answer is usually no longer correct.
If SO wants to be a knowledgebase, it needs to to be redesigned as it's current structure is not well suited to it.
Having an oracle that knows how to put a framework of events together (even wit errors) is much better than asking a human to do it from scratch.
When I asked him what prompted him to do that, he said copilot suggest it so I just followed. I wonder if you could hijack copilot's results and inject malicious code as many end users does not understand lot of the niche code it generates sometimes, you could manipulate them to add the malicious code to the org's codebase.
It might even take the context of the typos in your code comments, and conclude "yeah, this easy to miss subtle error feels right about here".
Now I'm wondering, can you put in a comment which the LLM will pay attention to such that it generates subtle back-doors? And can this comment be such that humans looking at the code don't realise this behaviour will be due to the comment?
Apparently, if you tried to access a class member without specifying a class instance, one of Eclipse's "auto-fix-it" suggestions was to make all members of that class static, and he just followed that suggestion blindly.
A significant problem is the subconscious defense mechanism or bias that compels us to conclude that AI has various shortcomings, asserting the ongoing need for status quo.
The capabilities of GPT-3.x in early 2023 pale in comparison to today's AI, and it will continue to evolve and improve.
https://news.ycombinator.com/item?id=27771186
Yet people don’t like it in this thread. Does it touch a nerve?
That is a great point. The issue of not asking the right questions has been around as far as I can remember but I guess it wasn't seen as the bottleneck because people were so focused on solving problems by any means possible that they never had to think about solving problems in a simple way. We're still very far from that though and in some ways we have taken steps back. I hope AI will help to shift human focus towards code architecture because that's something that has been severely neglected. Most complex projects I've seen are severely over-engineered... They are complex but they should not have grown to hundreds of thousands of lines of code; had people asked the right questions, focused on the right problems and chosen the right trade-offs, they would have been under 10K lines and way more efficient, interoperable and reliable.
I should note though, that my experience with coding with AI is that it often makes mistakes for complex algorithms, or it implements them in an inefficient way and I almost always have to change them. I get a lot of benefit from asking questions about APIs and to verify my assumptions or if I need a suggestion about possible approaches to do something.
Honestly I find this to be the biggest advantage to using a coding LLM. It's like a more interactive debugging duck. By the time I've described my problem in sufficient detail for the LLM to generate a useful answer, I've solved it.
In my code reviews the person who wrote the code needs to explain to me what they changed and why. If they can’t then we are going to have a problem. If you don’t understand the code that an LLM spits out you don’t use it, it’s that simple. If you use it and can’t explain it, well… we are going to have to have some discussions and if it keeps happening you’re going to need to find other employment.
The exact same thing has been happening for pretty much the entire time we’ve had internet. Stack Overflow being the primary example now but there were plenty of other resources before SO. People have always been able to copy/paste code they don’t understand and shove it into a codebase. LLMs make that easier, no doubt, but the core issue has always been there and we, as an industry, have had decades to come up with defenses to this. Code review being the best tool in our toolbox IMHO.
But that's not what these LLM systems are. https://hachyderm.io/@inthehands/112006855076082650
> You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to.
> Alas, that does not remotely resemble how people are pitching this technology.
It is exactly what happened to you: it wrote bullshit. Plausible bullshit but bullshit nonetheless.
This sentence summarizes the issue with the current AI debacle, along with the whole "just copy/pase code from stackoverflow and earn top bucks" meme that was going around in 2010s.
You're not gonna be a valuable dev if you're just write wrong code faster. Not only does chatgpt/copilot give haphazard code half of the time, it approaches seemingly random syntax and format. Even if LLMs are polished, you're gonna need stand software engineering knowledge to know what's right and wrong.
An year into the project I am forced to revise my opinion. When browsing my code-base I often stumble in abstruse niche solutions for problems that should not have existed. It was clearly the work of someone inexperienced walking through walls in an AI-fuelled coding frenzy.
Having an oracle that knows all answers is useless if you don't know what to ask.