I agree that it is basically just code suggestion software, or like a big autocomplete. But it's definitely being marketed as more.
Yeah, CoPilot already replaced my job. I used to be the office copypasta, going around to my coworkers desk all day long to fix their problems by googling them, then pasting in the first snippet to pop up. It didn't matter if the language matched or not, it was more the thought that counted. Now I've had to get a job as a loan rejection stamper at my local bank. It's not nearly as fun or exciting, but at least my effort makes a difference now. A big difference.
>Tests without the toil. Tests are the backbone of any robust software engineering project. Import a unit test package, and let GitHub Copilot suggest tests that match your implementation code.
Accompanied by a picture of copilot filling out the implementation of a unit test, based on the test's name.
The implication seems very clearly that it is more than a google search. According to the ads, it understands your code well enough to write the unit tests for it!
Yes, Github Copilot marketing has overstatements -- but it also has caveats about its limitations and creating errors.
In other words, the landing page has a mix of statements ... some hyping up the potential -- and some disclaimers to mention the reality and pitfalls.
Excerpt from that same Copilot landing page about it being wrong more often than right: https://copilot.github.com/
>How good is GitHub Copilot?
>We recently benchmarked against a set of Python functions that have good test coverage in open source repos. We blanked out the function bodies and asked GitHub Copilot to fill them in. The model got this right 43% of the time on the first try, and 57% of the time when allowed 10 attempts.
>Does GitHub Copilot write perfect code?
>No. GitHub Copilot tries to understand your intent and to generate the best code it can, but the code it suggests may not always work, or even make sense. While we are working hard to make GitHub Copilot better, code suggested by GitHub Copilot should be carefully tested, reviewed, and vetted, like any other code.
Right but realize how much of a marginal gain we're fighting for here. You can already do this with a separate window and some extra clicking and typing. Saving a little clicking and typing isn't a huge win, unless you're also willing to argue the most valuable work a SWE does is the clicking and typing.
This whole thing feels ultimately like a PR spot that we're all tripping over ourselves to participate with. Its the SWE equivalent of getting all excited about how self-driving is just around the corner TM. It's a premature waste of energy until real advancements are made.
The automatic SO search pane I was describing might give the same benefits. Most of the time maybe it would be useless. Sometimes it would show what I knew I wanted and save me the trivial effort of a search. The real gain would come from when it hit something that I should have asked but didn't think to. Then, I'd have a valuable insight I never would've had otherwise sitting in some pane ready for me.
Maybe it's just my imagination - but that's how I think it could be good.
I'm using it primarily for Swift and some Kotlin.
Another thing: I was using the free version in beta so it was effectively the full blown version. It worked super well. I changed jobs, changed hardware, and haven't set up my key yet (hopefully still grandfathered in) but it's felt not as good - the output is a lot shorter, for one. I would likely get the paid version if I lose my beta key. But it's weird that you have the paid version and it's not useful.
It's also possible they nerfed the models over time in the name of some other tradeoff, like cpu or memory. May still be able to configure it. But I distinctly remember it spitting out entire lines or chunks of code, exactly as I would have written them.
It works best when you are writing stuff similar to your own codebase. Perhaps try vendoring some Swift or Kotlin in your project dir to get the engine to fine-tune on it?
I also think the discussion around searching versus autocomplete is interesting. What we need is something that autogenerates a Google search based on your code context and shows you some promising results. I'm imagining a pane in VS Code getting populated with Stack Overflow answers whenever I slow down.