Agreed. How many times have you seen the "accepted" answer on a Stackoverflow be suboptimal if not outright wrong, and a much better answer is lower down the page. For me it feels like 50% of the time or more.

I think they kind of do. On the Telemetry section of the Copilot site, they mention that they relay whether a suggestion was accepted or rejected.

I wonder, how much better could Github Copilot become by also looking that the modifications that are subsequently made to the accepted suggestions? Obviously this would go quite a bit further in terms of telemetry, and may become an issue. They would essentially be training based on non-public code at that point.

I guess that's a start, but I think comments, flagging, would have value. There's lots of reasons you might not accept something.

> On the Telemetry section of the Copilot site, they mention that they relay whether a suggestion was accepted or rejected.

That will make it the target of a collaborative bias attack.

Wait, are we sure it'd need to be collaborative?

I'm guessing but surely it reads the code you end up with and uses that to train?

So it will be self-reinforced to get worse and rot over time? Since surely all code that used Copilot will be slightly worse than code without (extrapolated on the assumption that Copilot is wrong some of the time).