In my experience, it's a quality of life improvement, but the things that dictate actual time to market is bottlenecked by things that aren't solved by copilot, such as overall design, decision making, requirements gathering, code structure/architecture, solution ideation, user acceptance, infrastructure setup, etc.
I think if it eventually could help with those other tasks, you'd see time to market gains, and that would start to make it really valuable.
[0]: https://www.theverge.com/2022/11/8/23446821/microsoft-openai...
1) It's the code license that matters, and the depends on what license the code the developer is building is using
2) License compatibility is undecided (see eg the different views of Apache Foundation and FSF on the compatibility of ASF 2.0 and GPL)
3) It's easy for someone to deliberately produce code that violates a license. MS is a big target and you can bet license trolls would chase it on that.
Probably more reasons, but there's a good start.
Never ever I'm risking breaking copyright, and I also don't like Microsoft not including their own code in the model.
From allegations I have read across the Internet, Microsoft might be doing those who use Copilot a favor.
This is a serious question, I'm apparently just unaware of the horror-stories that can come out of breaking copyright. (Not from the US so..)
For me its most useful for helping with bash scripts and small simple stuff, just saves a huge amount of time I would spend googling and checking small things. Not sure if copyright is relevant there or not, it certainly isn't something I am worried about. Interested to hear what your fears are based on.
If you use code or art generated by an AI that was regurgitating training data, you can be sued for copyright infringement.
The way that AI gets training data these days is... questionably ethical. It's all scraped off the web, because the people who make these AIs saw court precedent for things like Google Books being fair use and assumed it would apply to data mining[1]. Problem is, that does nothing for the people actually using the AI to generate what they thought were novel code sequences or images, because fair use is not transitive.
[0] This won't work in current Copilot because a) I'm misremembering the comment phrasing and b) they explicitly banned that input from generating that output.
[1] In the EU, this practice is explicitly legal
But I find I get most pleasure out of using CoPilot for personal tasks, everyday stuff that isn't for work. Surely there is no issue there?
And to counter the note about the scraping sources-- I have recently (out of work) been experimenting with using CoPilot to translate open-source MATLAB code into python and zig. If I make something useful in this way, it will be open source and I will credit those who wrote the original code.
Surely a use-case like that last would have to tick all the moral and legal boxes?
"GitHub Copilot blocks your ability to learn." Is a common refrain.
I don't see ANY industry-wide consensus on whether GitHub Copilot truly helps developers right now.
The only scenario I can get anyone to agree on is generating templates. Aka, JSON or CSS files that you then edit.
Every time I don't have to context switch to look up some technical errata in my browser is a complete win for me.
I've never heard anyone claim it blocks the ability to learn - if anything it's the opposite. Many people like how it shows you APIs you weren't aware of.
But I did find that I need to turn it off at certain stages when learning (or re-learning) a programming language. It's seemed counterproductive until you have a good grasp on the basic syntax of the language. But the "showing you new APIs" does seem to be a thing that actually helps.
In general, you should not be accepting code completions that you don't understand. I'm usually stricter, in that I only typically accept completions that line up with what I was planning to type anyway.
It's good for learning new packages in Python.
It's quite poor for learning a totally new set of programming language grammar.
If you're a Javascript developer of 5+ years of experience, it's probably absolutely fantastic. Not so great if you're learning a new programming language.
- learning a new code base
- learning a new (popular) library
- learning a new language
You could compare it to, say, eslint but for other languages .
What is an idiomatic way of X with Y?
Well, copilot will give you the answer in 10-30 seconds less time than opening a browser and searching.
Im not going to argue it’s legal merits, but as a learning tool, it’s very much like having a smart linter.
The larger scale code generation is less obviously useful and usually wrong, I agree.
On the contrary, it frequently suggests code that adhere to better practices.
I haven't wanted to use it personally. I'm also not a senior dev or anything so my opinion is not worth much.
Others have already pointed out the case as a reply.
You don't fit that stereotype, of course. So feel free to ignore the following.
Developer tools have learning curves. One doesn't simply open vim/emacs for the first time with a full understanding of how to use it (or why it's a good tool to use, even). Historically, we have had _no_ problem with the steepness of this curve. But, when it comes to Copilot, there's a lot of "tried it and it output an obvious bug! how did this make it past quality assurance?? such a liability!" Just very reactionary and all-that.
Anyway, sorry for the toxic response.
Thanks for the rigorous examination of my comment though. /s
I kind of hoped that it would propose good tsx for my components (react+mui) but instead it’s basically useless. Maybe problem is that I chose “filter out exact code reproductions.”
Copilot has a ton of still unresolved legal and compliance issues (copyright violation problems, sending proprietary code to Microsoft as you are writing it, etc.) and most larger businesses won't touch it with a 10 foot pole for that reason. There is even a class action lawsuit against Microsoft over Copilot already.
And before someone jumps in: I and my other co-founder who also uses copilot (We are the only two in the company who do, I think, without checking) are the compliance team. We're both very senior and use copilot basically a line or three at a time as a smart autocomplete. It's still worth it.
I think this is the best way to think of CoPilot. GitHub is selling it like its going to write all your code for you, but in reality it is just next-generation auto-complete.
That's not a bad thing. In some ways I'd argue its actually better. GitHub needs to change its marketing because even most developers seem to think that its out there to take away our jobs. Its not and can not. But it provides the smartest auto-complete you've ever seen and that can be useful, especially when wading through mundane parts of your codebase.
I find typed languages like Rust or Typescript make VS Code super powered and provides much more value than copilot.
Indeed, I payed for a debugger because MSVC is pretty terrible
Yeah it's pretty dumb most of the time. But I know that, and I don't use code from it without carefully checking it out and modifying it. But it's still a huge help. Just the time saved writing tests alone pays for it. And I've had a few spooky experiences where it feels like it knows the bug fix before I do. Think of it as a smarter auto-complete.
The technology has a long way to go, but I completely disagree with Kite here. It's already good enough to pay for. If my company didn't pay for it, I would. I already pay for JetBrains, and it costs more than Copilot. I would give up JetBrains before I give up Copilot.
My guess here is Kite positioned themselves as a free alternative to Copilot and then couldn't monetize. There very likely is more to it though.
(Don't have much experience with it)
It also predicts comments and logging messages amazingly well (you type "logger." add 7/10 times get what you want, sometimes even better), incorporating variables from the context around. This speeds up the tedious parts of programming when you are finalizing the code (adding docs + tracing).
Honestly, Copilot saves me so much time every week while turning chores into a really fun time.
95% of tests are: instantiating a class, running a method, and then asserting that the result. Tests do not or should not be crafty creative code snippets. They are boring functional code blocks by design and most are very similar, only changing out inputs and assertions between tests.
I also don't apply DRY (don't repeat yourself) to tests. Tests should be independently readable beginning to end, no context needed. After all, the true value of a unit test is to take a block of code too complicated to easily fit in your mind, and break it down into a series of examples simple enough to fit.
Once you get the hang of how to prompt it (mainly through clever use of comments) it can be a HUGE time saver.
This, so much. My code since using Copilot is easily ten times better tested than it was before, and I wasn't especially lazy when it comes to testing.
Given 1-2 hand-written unit tests, Copilot can start filling in test bodies that correctly test what's described in the function name. When I can't think of any more edge cases, I'll go prompt it with one more @Test annotation (or equivalent in another language) and it will frequently come up with edge cases that I didn't even think of and write a test that tests that edge case.
(One great part about this use case for those who are a little antsy about the copyright question is that you can be pretty darn confident that you're not running a risk of accidental copyright violation. I write the actual business logic by hand, which means copilot is generating tests that only interact with an API that I wrote.)
Regardless, it is interesting to think about what domains are easier to generate effective models for. I would expect it to be easier to generate a supervised model <test description> => <test code>. My intuition is also, that it is easier to generate React component code, and harder to generate feature code.
And was all over the tech news at the time.
I kind of suspect you are intentionally trolling…
What was ruled by the supreme court was that Google's usage of the API (which had already determined to be copyrighted) fell under fair use in copyright law.
Kind of; the appeal denied was an interlocutory appeal (an appeal before final judgement), so the lower court ruling was left in place until final resolution of the case potentially to be settled in any final appeal.
However, while copyrightability of APIs was raised on the final appeal, the Supreme Court sidestepped it, ruling that because Google’s use was fair use even if the API was copyrightable, it was unnecessary to decide the copyrightability question at all. So the Federal Circuit decision remains in place on copyrightability.
On the gripping hand, though, that decision really doesn't matter much because Federal Circuit decisions on issues outside of those it has unique appellate responsibility aren’t binding precedent on trial courts (it is supposed to apply the case law of the geographic circuit that would otherwise apply, but its rulings don’t have the precedential effect that rulings of that circuit would have.)
So, basically, as far as appellate case law, Oracle v. Google provides no binding precedent on copyrightability of APIs, but precedent that at least one pattern of API copying is fair use if APIs are copyrightable.
Which isn’t encouraging for anyone looking to protect an API with copyrights.
It bases this generated test cases on other similar test cases in other software, including GPL licensed
Testimonials of this form are near worthless to a company. Maybe it's true for you. Statistically, it's highly likely to be misleading.
People overestimate their willingness to pay for something for a number of reasons, but one of the biggest is that they incorrectly visualise what the choice to pay or not looks like. They often imagine a moment of abstract choice after which everything remains exactly the same but some small amount of money magically vanishes from their bank account. In reality, paying for something is a tedious inconvenience, and not paying for it more often takes the form of never getting round to putting your card details in than consciously deciding "this isn't worth it".
It can be taken to questionable extremes, but there's truth in the idea that the only real evidence as to what customers will do is what they actually do, not what they say they will do. I don't know if their interpretation is correct, but it sounds like Kite at least has evidence of the former sort.
Especially when you don't market to developers in general, but freelancers/contractors specifically. It might be hard to sell to salaried developers (they'll buy because it's nicer to work with good tools), but it's easy to sell tooling to anyone who makes more money when they get more done.
Some IC developers will pay for tools, it’s very hard to have that happen at a price point that supports the scale required. So feature discriminate on the things their boss needs, and charge for that. And then the next set of features for their bosses’ boss, and so on until you’re selling into the C-suite.
I think this is probably true. If you need a tool for your day job, your company ought to be paying for it. Some companies have slush funds for small purchases like books, but subscription costs for services would normally need to be approved. If you're a solo consultant then perhaps you'd pay for tools that make you more productive. But for personal projects the value-add would have to be pretty high to be paying another O($10-20) a month on top of other subscriptions.
The big group of "hobbyist" coders are students, and they get copilot for free via Github's very generous edu package (and so does anyone with an edu email address I think). The bigger problem is that this is a very expensive project. It's better suited to a big company with money to burn and deep pockets to give it away to junior devs who will evanglise for it at their new companies (e.g. students) for nothing. See Matlab.
The sheer volume of subscription services I've signed up for as the CTO for a startup is mind-boggling. $8 here, $19 there, $49 for something important, $99 for something essential.
Some tools are easily worth it, especially when you see what is charged for other (less valuable) tools.
Gitlab, Confluence, Jira, Asana, 1Password, co-pilot, codepen, sentry, jetbrains, gitlab plugins for jetbrains, Visual Studio, Docker Desktop, Perforce, Slack, etc;etc;etc;etc
Then there's things like Spacelift ($250!)!
The most frustrating thing is that:
1) I need to justify these expenses each for what value they bring, some things are nice to have but bring so little value on paper.
2) You can't just enable tools for some people, there's huge overlap and that overlap gets greater
I get that people need to be paid, but these things very quickly add up. I'm paying about 7-13% of peoples salaries already in these subscriptions, and I feel like a total dick for saying no to people or trying to consolidate these.
Maybe, but most of the tools you listed are not in that category IMNSHO.
Ones I excluded for bring "not worth it" are Lens ($20/user/mo), Snyk ($139/user/month) and Postman ($36/user/month) - contrast those with Gitlabs pricing to understand the value trade-off.
You either care about licenses of dependencies or just decide copyright doesn't apply to you :D Seems that using both is a contradiction.
They have a bunch of rules for IaC that prevent default behaviour from biting you in the butt.
It is weird that software engineers are the only engineer-types that are supposed to be able to do their job with just a computer and a built-in editor.
Not really. We aren't actually engineers. Someone appropriated the title and misused it, and now there's no putting that genie back in the bottle.
I have yet to meet or work with one.
Engineering is expensive, and most software developers rightly default to a pragmatic craftsman's attitude. Build something people can use and iterate on it based on feedback.
When there aren't lives at stake, it's usually the right call.
It is not a coincidence that software development tooling is one of the only fields that has has been thoroughly eaten by open source. It's because the user-developer circle is complete. A mechanical engineer probably doesn't have the skillset to improve the CAD program they're running. A software developer probably has the skillset to improve VIM.
That said, I'm not even sure VSCode or Copilot is lucrative, if it wasn't owned by Microsoft, could they both be sustainable businesses?
Kite may have been the first to market but copilot blew them out of the water in terms of overall functionality.
Sounds like everyone has dodged a bullet!
Since we literally build all of this our B.S. detection meter is really high.
Kite thought it can go after the up and coming new developers by doing slightly shady things.
However, developers also have an incredible allergy to such tactics and it forever taints your brand.
So overall, developers do pay for tools, just not useless ones with shady growth tactics.
I briefly tried Kite a few years ago. I didn't notice anything shady although maybe I just didn't stick around long enough.
What shady tactics are you referring to?
An old comment from another user. https://news.ycombinator.com/item?id=14837253
Developers easily fool themselves thinking they’ll save $9 p/m by building something from scratch in 3 weeks.
Oh no. The only thing that's high is our conviction that our BS meter is high. We fall prey to, come up with and promote as much BS as the next person.
(I eventually stopped subscribing, in part because they were too slow distancing themselves from Russia, in part because of their movement away from open source with their newer tooling.)
Developers will pay for software, if the value proposition is there.
I'd cut them some slack here. They had to get their team out of there first—with the way Putin is running things, they sure as hell couldn't announce they were leaving Russia until everyone who was going to follow them was out of there.
On the day of the invasion they tweeted a statement condemning the attack, and within two weeks announced they were leaving Russia.
https://twitter.com/jetbrains/status/1496786254494670851?lan...
https://blog.jetbrains.com/blog/2022/03/11/jetbrains-stateme...
Two weeks was impressive, just not impressive enough to get me to renew.
A few tools that I put on the company card when I worked at a Big Tech Co as an IC:
* DataGrip (Jet brains)
* Colab Pro (Google)
* Postman Pro
These were all small $ enough where I didn't need to justify the expense. It was just assumed that if I thought the tool was worth the $, it was.
For more expensive purchasing decisions, there was a longer purchasing/approval process. But the expense would have to be 5-6 figures per year before hitting this barrier.
If you are an IC reading this and have never tried to expense a tool you find valuable, give it a shot. You might end up surprised at how much management appreciates the ability to trade money for business value.
* JetBrains (PyCharm professional, DataGrip, and Goland) ~$250/yr
* Lucidchart (Diagramming) ~100 /yr
* Paw (HTTP Client) ~$50 /yr
* Docker Pro ~$60 /yr
I think there's probably more, but I'm not at my work laptop to look, but those are the big ones. Those are only individual subscriptions. There's also huge costs when associated with things like Gitlab Premium ($20 - $100 /user/month), CI/CD, Code coverage tools, security scanners, etc. Companies pay A LOT for development tools.
If Kite thinks that the problem why no one will pay $9/mo for their service is because developers or their company's are cheap, they need to re-assess. The reason they couldn't sell their service is because it wasn't providing enough value to justify it. But companies are paying hundreds of dollars a month per developer in most cases for various tools. The extra $9 for Kite isn't the dealbreaker if there was enough value from it.
Frankly as a developer i've got more problems than i can count and if it involves a GUI i tend to prefer to pay for it. I love FOSS but UX is just not often a focus. I have better experience with paid products. Assuming the licensing isn't punishing.
Huge hurdle, obviously, but my thought was an expanded set of LSP features from JetBrains, disconnected from their IDEs. They spend quite a bit of time and money on developing DX but it's all inaccessible to those of us who prefer different editors.
I'd pay a lot for these sorts of features that go above the existing FOSS LSP, while retaining more integration to the FOSS tooling we've come to know and love (like editors or choice).
There's more i'm positive, but i just woke up :)
https://github.com/community/community/discussions/7553#disc...
https://www.reddit.com/r/vscode/comments/qromfk/is_there_a_w...
Maybe the product is poor. Maybe people didn’t believe the claim. But nobody said, “nah I don’t want my devs to be 18% faster.”
Without having tried it I'm assuming either their product was not good enough or their marketing department isn't strong. Developer really tend to neglect the latter.
[1] https://www.ciodive.com/news/github-copilot-microsoft-softwa...
The first group gets annoyed all the time because Copilot fails to write most code when prompted with comments, or writes inaccurate code at best. They get upset when they see that Copilot can reproduce GPL code when prompted in a specific way.
The second group most prompt Copilot by allowing it to tab a complete a line or two at a time, and they are actually super happy because Copilot is way better than any other existing autocomplete; it's basically in a class of its own. To them, the GPL issue seems a bit more abstract, because they would never use Copilot to do that anyways.
I fall pretty firmly into the second camp (can you tell?). Allow me to soliloquize for a moment. Copilot is an incredibly powerful tool, probably the most powerful one I have, but, just like any tool, you need to really learn your way around it, and understand what it can and cannot do, before you start making judgments. I'm not surprised that you turned it off after using it twice. Imagine saying you stopped using React after making two components!
Maybe I should write up a bit more about how I use Copilot, but in a nutshell I feel that it falls somewhere between a 2x-better autocomplete and (and this bit is even more interesting) a tool similar to google search, but more tightly integrated with the coding environment. The second bit is why it's so good. Imagine if I were to continuously google search everything I was doing while coding, while I was coding it. Sure, most of the time it'd just confirm you were doing the right thing, but... every now and then, Google might turn up a better strategy than the one I was currently trying. That's how I feel Copilot works all the time; it's continuously "google searching" for alternate approaches, and every now and then it'll be like, "aha, did you think of [this thing]" and really take me aback, because I wouldn't have even thought to Google for that particular bit of code / problem / strategy.
Of course, you could continuously google search everything you did as you did it, but it would be a massive waste of time. Just imagine Copilot is doing it for you, and returning what it found. Most of the time I know what I'm doing, but every now and then, the result is remarkable.
It is true that more commonly it suggested at most a few lines of obvious code which could really only be written the way it suggested, but a number of people in the comments on this article mentioned using Copilot to come up with test cases, so I think people are actually using it to suggest larger snippets of code.
I never used Kite, but I've tried Github Copilot twice, and found it marginal at best (and distracting at worst - which is why I turned it off both times). If Kite was similar, the reason I'm not paying is that coder AIs are not providing any value.
Developers are somewhat reluctant to pay for tools but I think you can get them to pay for things that are worth it. I've been paying for code editors for years.