News for hackers. Stuff that matters.
Re:AI Copilots Are Changing How Coding Is Taught by Taylor_OD on Saturday May 04, 2024 @12:33AM (#40254844)
> It was clearly the work of someone inexperienced walking through walls in an AI-fuelled coding frenzy.

Isnt this what code reviews are for? I catch a decent amount of code that looks AI generated. Typically, some very foreign pattern or syntax that this engineers never used nor is common in the codebase. Or something weirdly obtuse that could be refactored and shows a lack of understanding.

Normally I ask something like, "Interesting approach! Is there a reason to do it this way over (mention a similar pattern in our codebase)?" or if it's egregious, I might ask, "Can you explain this to me?".

This feel similar to early career engineers copy pasting stack overflow code. Now its just faster and easier for them to do. It's still fairly easy to spot though.

Re:AI Copilots Are Changing How Coding Is Taught by steve1977 on Saturday May 04, 2024 @02:47AM (#40255492)
> Isnt this what code reviews are for?

Wouldn't you need to have people with a proper understanding of the programming language and framework to do the code reviews?

Re:AI Copilots Are Changing How Coding Is Taught by fhd2 on Saturday May 04, 2024 @07:16AM (#40256747)
You would. And if the juniors don't really _write_ code in the languages you use, if they don't make mistakes and do research themselves, it'll take them a lot longer to learn them sufficiently to be able to do those reviews.

Listening to people talk in Japanese, I picked up enough to have some idea what they're talking about. I can't speak the language beyond some canned phrases. I certainly wouldn't claim to know Japanese. And I definitely wouldn't get a job writing books in Japanese, armed with Google Translate.

For programming, my solution to this right now is lots of pair programming. Really gives you a good idea of where somebody is at, what they still need to learn, and lots of teaching opportunities. I just hired a junior and we spend about 8 hours a week pairing.

Re:AI Copilots Are Changing How Coding Is Taught by Taylor_OD on Sunday May 05, 2024 @03:39PM (#40267541)
Yeah. Are we assuming 100% of the people working on this project are incompetent? I thought OP was talking about new people joining a project producing low quality code that was likely generated by AI.

Code reviews, done by existing more senior members of the project/team, should prevent this. If it isnt then the project has more issues than AI generated code.

Re:AI Copilots Are Changing How Coding Is Taught by arrowsmith on Saturday May 04, 2024 @03:23AM (#40255654)
> It's still fairly easy to spot though.

Or maybe it's far more common than you realise, and you're only spotting the obvious ones.

Re:AI Copilots Are Changing How Coding Is Taught by Taylor_OD on Sunday May 05, 2024 @03:42PM (#40267581)
Sure. If its generating 85% code that passes a code review and 15% of code that doesnt... I'll take those percentages. That is certainly higher than the average early career engineers pr.
Re:AI Copilots Are Changing How Coding Is Taught by adastra22 on Saturday May 04, 2024 @12:57AM (#40254973)
Not all organizations can afford code reviews.
Re:AI Copilots Are Changing How Coding Is Taught by AlexandrB on Saturday May 04, 2024 @01:05AM (#40255015)
If you're trying to do things efficiently you can't afford not doing code reviews. Only well-funded organizations can afford to write the same code twice after the first attempt has to be thrown out.
Re:AI Copilots Are Changing How Coding Is Taught by nox101 on Saturday May 04, 2024 @02:31AM (#40255416)
I work at a FANMG company and see the value of code reviews for th code we're working on. But, I shipped 20 commercial products before that with no code review, just play testing (games). Our velocity without code review was easily 2x what it is with. For those projects I think that would still be true whereas I'd never suggest getting rid of code review at my current job.

The point being I think it depends on the project, it's size, audience, lifetime, etc whether or not code review is a win. my gut says it usually is a win but not always

Re:AI Copilots Are Changing How Coding Is Taught by PoignardAzur on Saturday May 04, 2024 @03:43AM (#40255754)
That's super weird.

At the one company I was where we did code reviews, they took at most one hour per day. The reviewer would go through the changes, ask for explanations when they didn't understand something, suggest small modifications to fit the codebase's style/structure/vibe, and move on.

They definitely didn't take 50% of our time. Half my time wasn't even spent coding, it was spent thinking about problems (and it was a workplace with an unusually high coding-over-thinking ratio, because of its good practices).

Re:AI Copilots Are Changing How Coding Is Taught by 331c8c71 on Saturday May 04, 2024 @04:06AM (#40255853)
> At the one company I was where we did code reviews, they took at most one hour per day. > They definitely didn't take 50% of our time.

Imo introducing code reviews might very well decrease the velocity by 50% even if the actual time spent on code reviews is much less.

I see at least two reasons for this: 1) increased need for synchronization/communication 2) increased subjective frictions and mental overhead (if only for task switching).

Code reviews are not special in this respect, similar effects can be caused by any changes to the process.

Re:AI Copilots Are Changing How Coding Is Taught by strogonoff on Saturday May 04, 2024 @04:53AM (#40256030)
Thorough code reviews are expensive. When I review code, I have to know how to solve the problem well in the first place—that is, do all of the effort that went into writing the original code under review, minus time spent typing.

Less time may be required if you have good rapport and long work relationship with the person, but I would say halving productivity sounds about right if we are talking mandatory reviews.

Using an LLM to generate code means having to do the same. The longer the autocompleted text, the more careful you have to be vetting it. Lower predictability, compared to a person and especially one you know well, means that time requirement would not go down too much.

Re:AI Copilots Are Changing How Coding Is Taught by deergomoo on Saturday May 04, 2024 @08:36AM (#40257195)
> When I review code, I have to know how to solve the problem well in the first place—that is, do all of the effort that went into writing the original code under review, minus time spent typing

Agree 100%, even when LLMs aren’t involved.

Certainly not all code reviews are like this—in many cases the approach is fundamentally sound, but you might have some suggestions to improve clarity or performance.

But where I work it’s not all that rare to open up a PR and find that the person has gone full speed in the wrong direction. In those cases you’re losing at least a morning to do a lot of the same leg work they did to find a solution for the problem, to then get them set off again down a better path.

Now in the ideal case they understood the problem they needed to solve and perhaps just didn’t have the knowledge or experience to know why their initial solution wasn’t good; in such a scenario the second attempt is usually much quicker and much higher quality. But introduce LLMs into the mix and you can make no guarantees about what, if anything, was understood. That’s where things really start to go off the rails imo.

Re:AI Copilots Are Changing How Coding Is Taught by strogonoff on Wednesday May 08, 2024 @02:08PM (#40301190)
> in many cases the approach is fundamentally sound

Sure, but I mean you still have to determine whether it is sound or not in the first place; i.e., you must know how to solve the problem. You can only evaluate it as right or not if you have something to compare against, after all.

If you have worked with a person for a while, though, I can see that you can spot various patterns (and ultimately whether the approach is sane) faster thanks to established rapport. Not so with LLMs, as you probably agree.

Re:AI Copilots Are Changing How Coding Is Taught by PoignardAzur on Saturday May 04, 2024 @02:28PM (#40259389)
I mean, it's something we did once per day, usually towards the end of the day. This was pre-Covid, so our manager was in the room next to ours and we could just pop by his office to tell him we were ready for today's review.

It really doesn't need to be a massive amount of task-switching. And the benefits were obvious.

Re:AI Copilots Are Changing How Coding Is Taught by KronisLV on Saturday May 04, 2024 @05:19AM (#40256143)
> They definitely didn't take 50% of our time.

It probably also depends on the people that you're working with. I can easily imagine the velocity plummeting when the person reviewing your code loves to nitpick and bicker and ask for endless further changes with every next round of alterations that you do.

Doubly so if they want to do synchronous calls to go over everything and essentially create visible work out of code that already worked and was good enough in the first place.

I'm not saying that there aren't times like that for everyone occasionally, but there are people for whom that behaviour is a consistent pattern.

Re:AI Copilots Are Changing How Coding Is Taught by CuriouslyC on Saturday May 04, 2024 @08:35AM (#40257187)
OMG this. Nitpicking code reviews are the worst. I've actually instituted a rule that code must be formatted with an opinionated formatter and linted, and whatever the linter/formatter decide is right, and no formatting/linting comments are allowed on code review to avoid this sort of bullshit review comment.
Re:AI Copilots Are Changing How Coding Is Taught by CuriouslyC on Saturday May 04, 2024 @08:32AM (#40257170)
Code review matters when you have a product that has an indeterminate lifespan. When new people come on, they need to learn the codebase and code review both helps ensure a codebase that is easy for new devs to understand (because hard to understand code doesn't survive review) and also it provides an onboarding experience as new devs can be coached during code review.

"One and done" products like games where you're mostly fixing bugs and releasing new content without major overhauls to the underlying code probably only need code reviews for code that is complex/clever where bugs are likely to be hiding, beyond that QA is fine.

Re:AI Copilots Are Changing How Coding Is Taught by dkersten on Saturday May 04, 2024 @02:48AM (#40255493)
Not just thrown out, but also the customer relations when the bugs affect customers, the investigation to find the cause of the bugs, the migrations to fix any bad/corrupted data because of the bugs. Code review is a far cheaper way to catch errors, bad code, footguns and places where bugs might creep in than letting them into production and having them impact customers. Code reviews also help keep code tidier and cleaner which makes future understanding and changes easier. Small companies can’t afford to omit code reviews.
Re:AI Copilots Are Changing How Coding Is Taught by rixed on Saturday May 04, 2024 @01:43AM (#40255182)
I don't understand. Is code review supposed to avoid writing coffee twice?
Re:AI Copilots Are Changing How Coding Is Taught by aaronbrethorst on Saturday May 04, 2024 @02:19AM (#40255354)
I recommend drinking coffee before requesting a code review or posting on an online forum.
Re:AI Copilots Are Changing How Coding Is Taught by kragen on Saturday May 04, 2024 @02:01AM (#40255268)
usually 'organization' excludes projects consisting of just one person, and if there are two people working on a project that involves writing code, the cheapest amount of that code for both of them to be familiar with is never zero, though it is often less than 100% of all the code they write. so i don't think that 'not all organizations can afford code reviews' is true, interpreted in the usual way. maybe you can be more specific about the scenario you're thinking of?
Re:AI Copilots Are Changing How Coding Is Taught by adastra22 on Saturday May 04, 2024 @11:25AM (#40258338)
As mentioned in the other comments, code reviews are typically not done in the game industry, for example, or other write-once, never read scenarios. Very early stage startups typically don’t have formal code reviews either.
Re:AI Copilots Are Changing How Coding Is Taught by kragen on Saturday May 04, 2024 @04:40PM (#40260122)
while that is true, and some of it is rational, other times it's just another stupid way people waste money and fuck up their projects. because even in a startup or a game there is code that people waste time debugging alone, or waiting for the one guy who knows it to get back from vacation, or out of bed

reviews don't have to be formal to be reviews

Re:AI Copilots Are Changing How Coding Is Taught by seattle_spring on Saturday May 04, 2024 @02:15AM (#40255338)
Can you elaborate on some scenarios you are describing? The only organization I can imagine that can't do code reviews would be organizations of 1. I don't even see how performing code reviews could ever be a net negative in terms of money nor overall productivity.
Re:AI Copilots Are Changing How Coding Is Taught by ben_w on Saturday May 04, 2024 @02:32AM (#40255425)
Not OP, but I've been in a few organisations with many devs but also where I was the only iOS dev — the other devs being Android, back-end, or web.

I would have preferred to have someone to collaborate with at each of those places.

Re:AI Copilots Are Changing How Coding Is Taught by oytis on Saturday May 04, 2024 @05:07AM (#40256077)
I think it should be possible (and encouraged) for an Android developer to know enough about ios to do review and vice versa.
Re:AI Copilots Are Changing How Coding Is Taught by JTyQZSnP3cQGa8B on Saturday May 04, 2024 @08:19AM (#40257096)
Or pair-programming reviews. It doesn't matter if you don't know a language, it's still very useful.
Re:AI Copilots Are Changing How Coding Is Taught by sensanaty on Saturday May 04, 2024 @10:35AM (#40257991)
Even if you're not working in the same language/framework/whatever, you can still do code reviews. You might not know all the syntax and high-level details, but even just an overview of the logic happening in the code is already better than 0 review of any kind
Re:AI Copilots Are Changing How Coding Is Taught by sensanaty on Saturday May 04, 2024 @10:35AM (#40257992)
Even if you're not working in the same language/framework/whatever, you can still do code reviews. You might not know all the syntax and high-level details, but even just an overview of the logic happening in the code is already better than 0 review of any kind
Re:AI Copilots Are Changing How Coding Is Taught by littlestymaar on Saturday May 04, 2024 @02:02AM (#40255272)
No organization can afford not having code reviews.
Re:AI Copilots Are Changing How Coding Is Taught by adastra22 on Saturday May 04, 2024 @11:26AM (#40258352)
No blanket statements apply to all organizations.
Re:AI Copilots Are Changing How Coding Is Taught by littlestymaar on Saturday May 04, 2024 @12:07PM (#40258587)
“No organization can work in absence of oxygen” proves you wrong.
Re:AI Copilots Are Changing How Coding Is Taught by Taylor_OD on Sunday May 05, 2024 @03:40PM (#40267555)
Those organizations shouldn't complain about code quality.
Re:AI Copilots Are Changing How Coding Is Taught by vouaobrasil on Saturday May 04, 2024 @07:17AM (#40256750)
> Isnt this what code reviews are for?

There is no substitute to doing something correctly in the first place. The problem is that in the real world, deadlines and lack of time will always cause the default solution to be accepted a small percentage of time even when it is not ideal. The increasing creep of AI will only exacerbate that and most technophiles will default into thinking of a new and improved AI tool to help with the problem, until it will be AI tools all the way down.

No thanks.

Re:AI Copilots Are Changing How Coding Is Taught by CuriouslyC on Saturday May 04, 2024 @08:28AM (#40257151)
I would agree with there being no substitute for doing something correctly the first place, but I would argue that in this case the "first place" is hiring/training better so your employees don't try to throw unrefined AI shit at the walls, but instead take the output of AI and hone it before creating a PR.

If you have an engineering culture that doesn't emphasize thorough code review (at least of juniors, leads and architects emergency-pushing is a different story) that's a problem. In addition to catching bugs, that's a major vector for passing on knowledge.

Re:AI Copilots Are Changing How Coding Is Taught by vouaobrasil on Saturday May 04, 2024 @10:24AM (#40257912)
I agree with you, no doubt there. But still, technology often offers the path of least resistance. So even if you want to hire better people, what of all the people who will grow up training themselves with Copilot? Yes, you can filter them out but it will be harder and harder to find good people. And then, companies will want to go after the bottom line: what if they can get a bigger and more complex product out there with AI assistance but with code that is more unreliable in the long term?

I tell you, it's spiralling out of control. Besides, even if you're doing the hiring, you may not have control if there is a profit motive. Us technical people cannot control the race to the bottom line, especially over the period of decades.

Re:AI Copilots Are Changing How Coding Is Taught by ungreased0675 on Saturday May 04, 2024 @09:14AM (#40257421)
>Isnt this what code reviews are for?

A foundational concept of quality control is to not rely on inspection to catch production defects. Why not? It diffuses responsibility, lets more problems get to the customer and is less efficient than doing it correctly to start with.

Re:AI Copilots Are Changing How Coding Is Taught by throwup238 on Saturday May 04, 2024 @12:26PM (#40258671)
Code review isn’t inspection, that would be testing - whether automatic or manual.

Code review is the feedback mechanism that allows quality control to improve the process, teaching the engineers how to do it right in the first place.

Re:AI Copilots Are Changing How Coding Is Taught by bcrosby95 on Saturday May 04, 2024 @10:36AM (#40257995)
It's harder to read code than write it. It's even harder to read code than use an LLM to write it for you.
Re:AI Copilots Are Changing How Coding Is Taught by Taylor_OD on Sunday May 05, 2024 @03:43PM (#40267583)
Yup. That is why code reviews are important... Read and understand the code that is going out before it goes out.