Now I do understand many won't do so. But they already do the same, just slower, with their current method.

I admit, there few times I copied but I can count on fingers the number of times I used something from stackoverflow in my entire career (I'm not counting situations when I read somebody's explanation, but didn't use their code, which is just an example of how it works). From posts that I see people write as if they do that daily.

Also SO is only an example. There are many sites, and I call BS anybody who doesn't Ctrl + C once in a while, be it from the docs.

And if you want others to grow, make sure your answers give knowledge without giving solutions.

Maybe it's the language you work with?

If I copy and paste React code from SO it'll 100% break my app due to incompatible abstractions, so it doesn't make any sense to do it. However if I need a complex generic Typescript utility type, copying things verbatim is usually fine.

I've never had the luxury, I mostly work on low level stuff and at best, SO is usually only helpful in pointing me in the right direction.

And still I copy paste.

* Just to clarify, I'm not saying SO is bad, but just specifically the practice of blind copy/paste without reasoning about & understanding the code you've copied. SO moderators encourage/semi-enforce descriptive answers for this reason; to add context to the solutions provided.

Even if you just limit it to people largely taking solutions wholesale from SO, I still think that it’s a good jumping off point. Of course it’s a mistake to not make any modifications or look deeper than the whatever is in the snippet, but the snippet is often much better than what a noob would come up with.

Also, it’s an opportunity for learning new patterns that you might not have come up with yourself.

I would respectfully disagree on this point. Anything that perpetuates doing this in any way will always have a negative impact on code quality. If an engineer is copying solutions wholesale, even if those solutions are robust and high-quality, that's an indicator of the approach they have to producing code on a daily basis, which is going to have a much larger overall impact than that 1 answer on SO.

SO is imo a net positive benefit to the community, but only by virtue of them doing other beneficial things that balance out with the harm of copypaste programming. But I don't buy that copypaste programming is benign.

> Also, it’s an opportunity for learning new patterns that you might not have come up with yourself.

Blind copypaste is by definition not an opportunity to learn, because you need to understand (hack/fork/adapt answers given) to learn from them.

Also why all the anti-copilot seems to think their code is great, or even well understood by themself.

I have counterexamples everywhere around me all the time for these 3 points.

"Blind copypaste" is just slightly more specific thing than just "copypaste". Copypasting code you fully understand and trust is fine (though in practice, is the rarer case). "Blind copypaste" implies you know it works(ish?) and don't care as much about the how.

> Also why all the anti-copilot seems to think their code is great, or even well understood by themself.

My code is certainly not great, but I like to think I understand what I've written pretty well (there are many contributors to poor quality code, not understanding what you're writing is just one potential one).

I also like to think that, while it's not great, it's on average going to be better than code put together by someone who doesn't fully understand what they've written.

> I have counterexamples everywhere around me all the time for these 3 points.

Do you? By what metric do you consider them counterexamples?

Copypaste programming doesn't have to be benign in order to be better than the likely alternatives. The people who blind copy/paste are likely not producing high quality code in the first place. In which case, blind copy/paste is often an improvement.

1. Delve into the original source to see what happens (or reverse engineer the binary) -- very time consuming!

2. Guess. Maybe write some test apps to play around with it till you get it working. This used to be very common, but leads to situations like the PS2 encryption algorithm being completely broken because the devs didn't understand the IV parameter.[1]

3. Go on StackExchange and find an example of what you are trying to do, usually with some helpful discussion about the parameters.

[1] You would think security libraries would have the best documentation because it's so important to get it right and difficult for the developer to detect mistakes, but I've found the opposite to be the case. They're some of the worst offenders for just showing you a cryptic function prototype and assuming you know everything about the underlying math already. It feels like the crypto guys think if you couldn't write your own crypto library then you aren't good enough to use theirs, kind of missing the point of libraries.

I wonder, how much better could Github Copilot become by also looking that the modifications that are subsequently made to the accepted suggestions? Obviously this would go quite a bit further in terms of telemetry, and may become an issue. They would essentially be training based on non-public code at that point.

That will make it the target of a collaborative bias attack.

I think context is important. It's not inherently bad. If you are inexperienced or don't know what the code/command is doing then yes that's not ideal.

But competent developers using Stack Overflow (and all other external memory devices) appropriately to aid their workflow is perfectly valid.

I rarely copy/paste verbatim from Stack Overflow (apart from perhaps bash commands where the answer is literally the exact command I need - and in that case, why would I not use it?). But I do copy/paste, as long as I understand the code, and then adjust it accordingly.

In my experience of coaching junior devs. The number one skill I've had to train them in, above all else, is the ability to efficiently search for and find answers to their questions/unknowns quickly. (As well as digest those answers and incorporate the knowledge into their understanding - not just blind copy/paste).

I'd go as far as to say that If you are a developer in 2021 and not constantly looking up things for reference (whether to jog your memory or to quickly learn new concepts) then you're either a genius with a photographic memory, working in a very constrained domain that isn't stretching you or you're just plain doing it wrong. :-)

There's always almost some nuance that the accepted answer might be lacking or you should know about that is mentioned.

Why not? Most of the software is not built by checking the formal verification of specification but by looking into the code and having reasonable understanding that it works. Also there will be errors in the code whether we use copilot or not. Personally if I don't have an option to do a build and run and look into the output, I am reasonably sure that there is at least one bug in something like every 50 lines.

Copilot makes it a method of FIRST resort.

If I know what I'm doing, and know I should escape my inputs... but copilot barfs up a block copied from someone who didn't... now I have to retrain myself to spend time reading the suggested code for stupid shit like that versus just using my own knowledge in the first place.

That's a far cry from "i know the method name I want has 'string' in it somewhere"-style autocomplete.

You're basically importing a library of "every stupid crap anyone committed to github" and just hoping the the library functions in it are safe and reasonable. That's a crazy dependency to bring into your project if your own programming skills are above that of the median github user.

So, roughly half of programmers would be _better_ served just blindly using Copilot's suggestions then?

Personally, I find that I work with so many different things so often that I "googling" is often much quicker even than reading documentation or even searching my own existing code.

But I also have _zero_ interest in Copilot at all, so what do I know?

A long command is almost never a good answer.

Even that cannot really protect me from missing things like the security implications that this blog post talks about.

In this case, I probably wouldn't have fallen for the stuff CoPilot (or an equivalent SO answer) suggested, as I learned about these kind of injection issues a long time ago, but there are certainly a lot of areas where I would fail miserably to detect subtle (security) bugs. APIs are full of subtle and often non-obvious pitfalls, and even algorithms can be, where the algorithm seemingly works except for non-obvious edge cases that you might not have considered.

Is this serious? I've never literally copy pasted from stackoverflow. I read the answer and then go write my own code. Did you seriously copy pasted an entire chunk of code and committed it to your codebase after some edits?

A tool like this has the potential to help good programmers work more quickly, but it carries the risk of acting as a crutch. A lot of people might never put in the work to become good programmers because they learn how to be just good enough at using Copilot to fake it.

In a world where there are lots and lots of people trying to gain entry in the software industry, there's a major risk of this leading to a lot of incorrect, and even dangerous code making it into production because nobody really had a look at it.

However, you seem to be making the argument that is good behavior. I say it is not. The time saved by automating the googling/copying/pasting is miniscule compared to actually understanding the context in which the code is suited and ill suited. That is the developer's work, and it isn't fed by only the code.

Developing ain't typing. It's solving problems. Most of the time the problem isn't in code. Even when it is in code, there's nuance in how to best solve it that isn't. The idea that AI is useful in understanding the real world context of the problem better than the human (who has the whole context, in code and outside of it) is naive, or disingenuous.

The code i produced during my first few years isn’t something anyone should aspire to.

Additionally you likely learned over time that most SO posts are not great but provide a good starting point.

I think that when $PROJECT lets everybody do something at a scale that was before impossible, even if it's the "same thing", it is not "the same thing".

Then when every program is eventually upgraded to this status, all of the code we run is bootstrapped through proprietary AI - including the AI itself - and it's black boxes all the way down. Programming is now an experimental science where we gather evidence, test hypotheses, press the "copilot" button and tune parameters, like evolutionary biologists studying DNA code.

That doesn't seem sustainable. It also seems like a poor cost to value ratio.

That doesn't seem sustainable. It also seems like a poor cost to value ratio

Those "prove" you see in academic paper are very specific case.

And I don't know why you put "prove" in scare quotes. There is formally verified software that has been proven to conform to a spec. They aren't just toy programs either, see SEL4 or CompCert for 'real' programs.

Fully verified programs do exist (e.g. CompCert C compiler), although for various reasons, it's still a niche.

> Those "prove" you see in academic paper are very specific case.

Not to put too fine a point on it, but every program written is a very specific case, so I'm not sure this is such a convincing point.

As you say, there is absolutely, provably, no general-purpose algorithm out there that will prove program correctness automatically. That is in no way to say that humans can't or shouldn't try to prove some programs correct, or even that you can't have an algorithm that will prove certain classes of programs correct.

With that said, I do also think your parent:

> We have figured it out. People with a PhD and given enough time can do it. Scaling this will be possible, especially with the help of AI, as soon as there is the actual will there to do it.

is too optimistic, and throwing a little cold water on that rosy prediction is appropriate.

Aren't we already there outside of very specific special applications? At the very least you have to assume every library you are using and the framework you are running on is correct. Sure, that works 99.9% of the time. If your testing framework can get the rest of the code to 99.5% of the time, is the .4% that large of a deal in a case where the other .1% is not?

When we look at what the market wants, what individual people want, do they want things proven correct given the increase in cost it brings? They may say they do, but spending behavior doesn't seem to align.

pretty much every domain, if you exclude finance & industries that intersect with life and death decisions such as pharma/medicine/airlines/defense.

what’s the correct order of movie recommendation ? its easy to see that given your past netflix history, there is an obviously incorrect order. but there is no obviously correct order - any number of orderings will work. correctness is not paramount.

what’s the correct puzzle to assign to a 1400 on chess.com ? obviously there are hundreds of them that would work. correctness is not paramount.

what’s the “correct price” of a used Ford Focus ? depends on whether you are a bandit who needs the car for a rapid getaway, or whether you are the brother in law of the used car salesman, in which case the correct price is zero usd.

the sole reason why 100 million developers crowd the web programming circuit and not other gatekeeped domains is because correctness is not paramount. whether your color code is off by a shade on this browser or your pixels misaligned on that browser, its all fine so long as it somewhat works. correctness is not paramount. otherwise nothing would ship.

Note that people die if data is lost, causing companies to go bankrupt (suecide). But really, not everything has to be correct, look at Intel and AMD. They've been compromising correctness for speed for quite awhile, and we're mostly fine.

EDIT: I assume from your comment that you formally verify every line of code in every system you depend on, and run RISC-V to make sure the hardware doesn't tamper with your instructions.

I suspect this to be the majority of code written.

I would agree with you if it was called "GitHub Self-Coding" where you are the destination clerk and letting the tool code. But that's really not the goal of the tool. Don't put it in hand of non programmers

The problem with Copilot is that it works just enough to be dangerous: it streamlines copy-pasting of unchecked code, but the code it offers has subtle bugs. It's even worse than copy-pasting from StackOverflow, because code on SO got at least a passing review by someone who understands it. Here, you get code generated from ML models. Unlike generating pictures of faces or kittens, when an occasional artifact doesn't matter much, an "artifact" in code that you won't notice will still make the code wrong.

> Don't put it in hand of non programmers

Putting it in hands of programmers isn't really any better. To make it work, you need programmers to be disciplined - more disciplined than they were when copy-pasting from SO.

It's like copying from StackOverflow while ignoring upvotes, comments, and anything but the first revision ;=)

Also the problem isn't code that's obviously wrong when you read it. The problem is when code looks OK, but is subtly wrong. Which keeps happening - as we know today, at least two examples featured on Copilot homepage have this problem. To make this work, you have to read each snippet super carefully - which defeats the whole point.

It's all well and good to say there should be something better than just discipline but there's no idiot-proof way of writing programs.

1. As a way to specify a problem and get a suggested solution (or a selection of them), to use as a starting point.

2. As a way to specify a problem, get it solved by the AI, and consider it done.

I'd expect that any developer worth their salt will do 1. I expect that of myself. I also worry this is so streamlined that people, including myself, will naturally shift to doing 2 over time.

This is similar to the problem with self driving cars - you can't incrementally approach perfection, you have to get it right in one go, because the space between "not working" and "better than human in every way" is where self-driving is more dangerous than not having it. When it works most of the time, it lulls you into a false sense of security, and then when it fails, you aren't prepared and you die. Similarly, Copilot seems to be working well enough to make you think the code is OK, but it turns out the code is often buggy in a subtle way.

> familiarity with language ecosystem

This is an interesting angle to explore the topic. Familiarity is a big factor when inspecting such generative snippets. For example, I'm really familiar with modern C++, and I'm confident I could spot problems in Copilot output (if and when it starts producing C++), maybe 50% of the time. If it's a logic issue, or initialization issue, I'll spot it for sure. If it's a misuse of some tricky bits of the Standard Library? I might not. I make enough of those mistakes on my own. Or, I know enough JS to be dangerous, but I don't consider myself fluent. I'm definitely going to miss most of the subtle bugs there.

On your other point, it's true that if you're working in an unfamiliar ecosystem, spotting subtle issues will be harder. But I think getting automatic suggestions that are probably idiomatic in that language will be more helpful than hurtful.

That still doesn't _necessarily_ imply that 'partially self-driving cars' are worse than actually existing humans. Really, anything that's (statistically) better than humans is better, right?

I don't think it's reasonable to think that even 'perfect' self-driving cars would result in literally zero accidents (or even fatalities).

As others have pointed out, at last Stack Overflow comes with context, the ability to rate and comment on suggestions, or even have a discussion about an approach. With this you take it or leave it. Saying you should already know what you need to do and what the tradeoffs are and how to evaluate the quality of the suggestion is basically saying this should have no value to you, and any consequences are all your fault if it does.

If you're using it to create something you don't know how to do then yeah you're in for a world of disappointment.

HN seems to be of the hivemind that random Joe Nocodes will be firing up VSCode and asking it for a custom version of Uber which.. yeah is laughable and honestly seems pretty obvious that that wont work.

Interesting metaphor. Notice how the Bash tab completion is a great tool, that increases people's productivity by a large amount, and receives only complements from anybody you ask. At the same time, the newish Windows cmd tab completion is a great productivity destroyer that gets mostly only criticism and you will find way too many people blaming it for losing data.

Do you know what is the difference? The bash tab completion is always correct. If it can't be correct, it gives you as much of a correct answer as it can, and leaves the rest up to you. The cmd tab completion doesn't care at all about being correct, it will always give an answer, no matter what you ask for.

If autocomplete on IDEs were that respectful, I'd like them too.

(But I also have no interesting in using Copilot myself, tho maybe I should try it myself now, if only on some toy side project.)

In fact, the copilot is just a normal pilot with the only difference that the pilot is also the captain on board, responsible for the police and security on board. And most of the times, companies choose who is the pilot and who is the copilot randomly on a per-flight basis.

So no, you wouldn't a copilot that gives subtly wrong information to the pilot (and vice versa)

What companies are you aware of that do this? The proper terms are "Captain" and "First Officer" and they are actual rankings within the company, not something that is randomly chosen on a flight-by-flight basis. The actual details of who does what during the flight are generally not related to the ranks ("pilot flying" and "pilot monitoring" duties can and do switch during flight) although the Captain is always the ultimate authority and will be the one to take control in tough situations because he's got more experience.

Typical (i.e. almost all) commercial flights will have a Captain sitting in the left seat and a First Officer sitting in the right seat.

Who's going to stop them? There's an army of non-programmers trying to break into the industry. You can bet they are going to get their hands on this.

How is this approach "wrong"?

Finding a bug in something which seems to be correct can be harder then writing the correct code yourself.

Especially if you might not (yet) fully understand what you are doing.

So Copilot is only a good idea if you are a experienced programmer with the discipline to put any auto generate part through a proper (ad-hoc) code review.

At the same time it looks especially appealing to someone just starting to learn coding...

Spotting bugs in code that has been generated by a GPT-3 derivative, with all the subtle mistakes that implies, is going to be even harder.

I'm kind of skeptical! I think your claim is reasonable tho so maybe I'm more skeptical of your confidence?

I'd love to read a follow-up from you after you tried using Copilot for an extended period; even (or maybe especially) if it's as bad, or worse, than you expect!

But copilot isn't even equivalent to a code review: code review is not only checking for correctness. It's also asking questions and helping the author walk through their solution by having them reconsider it. Copilot doesn't ask questions, nor can it answer them or provide a rationale.

I don't think I understand why this should be true.

- get multiple answers

- comments on the answers

- up/down votes

- explanations along side of the answer

and I still would argue you should never copy from stack overflow!! Instead understand why the answer is correct and then write your code based on that understanding, even if it produces the exact same code in the end.

a) I'm familiar enough with the code/topic that I'm able to judge whether the suggestion is good, or

b) I'm not able to judge, and need to consult other sources.

In case a, I could have just written the code myself. In case b, I still need to read up, so the potential gain is that Copilot gave me a hint as to what I should search for. But even that hint can't be trusted - there was another comment where Copilot suggested to use MySQLi in PHP, which has been the wrong choice for a long time.

So if the suggestions need scrutinization, the target group I see is people who need to write a lot of code in a familiar area, and are limited by the speed at which they type. This is arguably a very small set of people, and is unlikely to be the main user group of this tool.

Your job as a programmer is to ensure the correctness of the code you write, Copilot isn't really changing that.

If you don't have these systems in place, you're getting what you deserve. Hiring subpar talent and not having processes in place isn't Copilot's fault.

This is almost like blaming the car for an accident. If you have a shitty driver and the traffic lights aren't working, it's not your Corolla's fault.

It shouldn't be that way, but it's how people work, so we should expect it to happen. (Psychologically, if an action is very rarely rewarded, people are less likely to do it.) Even if you want to be vigilant, it will be a bit of an uphill battle to make yourself do it.

Also, checking the auto-generated code takes time. You, as an individual contributor programmer, may believe that the time is absolutely worth it, but you will need management to support that. This technology creates additional opportunities for managers to say, "Don't worry about that. Ship it." Which is another thing that really shouldn't happen but in reality does.

> Of course, when the correctness of your program does not matter in the first place, but just its output

1. There are in fact very, very few projects that try to prove their correctness. Usually, those are extremely critical or dependencies for potential critical applications. Even if a project does this they're doing it partially just for keeping ROI sane. Please correct me if I'm wrong but AFAIK, even most programs on aircraft don't use this approach, although they're definitely interested.

2. For most of the projects, the primary goal is to be useful, to be correct is at most secondary.

3. A program written by humans is not inherently correct, it's more likely to be the opposite. No matter it's written by a human or a machine, you should always write test cases to reflect the properties you care in your program. Properly tested programs written by the machine don't make it less correct than those written by a human.

I'm generally interested in Copilot, not because of the novelty it claims to be, but the value it might provide. I see it as a potentially useful tool just like IntelliSense but with greater potential.

At the end of the day, it's an AI "Pair". I've been doing pair programming for years, one of the lessons I learned is one should not assume anything just because he/her pair partner is doing it - both of them should be responsible for their program.

OpenAI leadership knows that there is just sooo much value to capture in more automation, even if it takes billions to get there. They also know, that there is sooo much money around not knowing what to do and so much excitement in the field generated by bystanders, not understanding a single thing.

Perfect setting for ride and reap.

I repeat: There is nothing open about this, the research scientists are being used in order to propel just another get rich sooner-than-later scheme (how should they know better, they are innocent scientists). Happy bulldozing what is left of society in order to generate profits and to shove the AI BS down the managerial class, because that's all just how the system works.

And, if know all that, you can exploit it, hacker.

When people make security corrections to those snippets, ideally it would learn the corrections. Perhaps even have a feature to flag a snippet as insecure, to help that process.

Copilot is imperfect, yes. But there is a large grey area between perfect and "utterly wrong"

Not on internet discussion boards.

So the problem is that Copilot will only work for 99.99% of all the code that's ever written?

I think that's OK. Most code is subtly bugger no matter where it originated from. It's always going to be up to the developer to check and test what they're committing whether it's from Copilot, a co-worker, or they wrote it themselves. I think Copilot is meant as an aid to the code writing process, not a replacement. It's not really that different to any other tool.

Plus, fortunately, most code is never subtly buggy if you look at small enough parts. If it doesn't work correctly it's very obviously wrong. As Copilot is aimed (for now) at small functions any bugs will manifest in clearly broken outputs. It's when Copilot can generate entire classes, or whole applications, that we'll need to reassess if the code it creates is actually worthwhile.

Just like Tesla’s Autopilot, this was poorly named, at a minimum.

The problem exhibited in TFA is not so different from people copy/pasting StackOverflow answers without understanding them.

If we start by assuming the developer is a "good dev", I can't imagine Copilot is going to do anything useful for them, as this isn't going to be a useful way for them to come to understand how the system they are using works, and it won't support them building abstractions to reduce boilerplate. This tool is simply not useful for them.

Which leaves us with the idea what Copilot is a tool designed for "bad devs" to be able to do less work to do the bad things they do, and while I sort of appreciate the idea of "it is worthwhile to support people who don't know what they are doing in their attempt to do the bad thing they are doing anyway", I have severe reservations about doing that with engineering... it at least needs to be marketed as that!

Otherwise, the two main things we should be doing for "bad devs" is either helping them become "good devs"--which this doesn't do for much the same reasons it isn't useful for "good devs"--or we should honestly be trying to convince them not to be a developer at this level at all (which might include building higher-abstraction systems that are easier to use and understand).

There's also a large difference in scope between the kind of suggestions a linter gives and what copilot does. Linter usually works around the syntax for equivalent semantics, while copilot tries to infer semantics.

The fact that there are so many people that like them that all the IDEs have the feature on by default with the off switches scattered all over their configuration so you can't turn them off is scary.

Having said that, I don't know how the much more integrated write/review/edit cycle will work in practice when using copilot. I don't think it will be the same as a junior developer/pair programming partner in any real sense. My initial reaction to copilot is negative, but I'm open to being proven wrong about it.

- The snippet you get from Copilot is generated by an DNN model. I'd think that alone should scare people. DNN models are correct on a continuous scale, not discrete. A bad pixel on a generated image, an extra comma in generated text, are considered OK. That kind of correctness framework doesn't work with code.

From what I read, Codex (the model powering Copilot) is a GPT-3 derivative. Surely you've played with GPT-3 before, you've seen the types of mistakes it makes. In my eyes, this approach just won't fly for code.

Empirically, this is false. There is an incredibly high amount of bad, incomplete, or vulnerable code on SO.

I'm not worried about it suggesting subtly wrong code. I write subtly wrong code all day every day, and sometimes I catch it, and sometimes I don't. I write tests, I use types, etc. I'll use the same tools to reduce bugs from copilot that I do for myself.

I'm not really worried at all.

I think it's an excellent idea. People who copy-paste from Stack Overflow will also copy lots of shitty code or use it incorrectly.

But those of us with a clue, have a neat prototyping system where we can modify the solution until it's coherent. It's still on us. Copilot never claimed to produce the best code. And doesn't have to. That's the whole point behind the "pair programming" metaphor.

Those are way too many words to say "never".