News for hackers. Stuff that matters.
Re:Kite is saying farewell, and is open-sourcing all of its ... by eloff on Sunday November 20, 2022 @05:00PM (#33685888)
My experience with copilot has been very different. It easily pays for itself, and getting my employer (seed stage startup) to spring for it for the entire team was an easy sell.

Yeah it's pretty dumb most of the time. But I know that, and I don't use code from it without carefully checking it out and modifying it. But it's still a huge help. Just the time saved writing tests alone pays for it. And I've had a few spooky experiences where it feels like it knows the bug fix before I do. Think of it as a smarter auto-complete.

The technology has a long way to go, but I completely disagree with Kite here. It's already good enough to pay for. If my company didn't pay for it, I would. I already pay for JetBrains, and it costs more than Copilot. I would give up JetBrains before I give up Copilot.

My guess here is Kite positioned themselves as a free alternative to Copilot and then couldn't monetize. There very likely is more to it though.

Re:Kite is saying farewell, and is open-sourcing all of its ... by jascination on Sunday November 20, 2022 @05:06PM (#33685936)
Out of interest, how are you using it to write tests? Do you just write "make a test for functionX" or something?

(Don't have much experience with it)

Re:Kite is saying farewell, and is open-sourcing all of its ... by premun on Sunday November 20, 2022 @05:38PM (#33686249)
It is amazing for typing out mock data. Say you're testing parsing of XML - it can easily suggest the the assertions over the data parsed from the XML. Example test that was 95% coming out of Copilot: https://github.com/dotnet/arcade-services/blob/61babf31dc63c...

It also predicts comments and logging messages amazingly well (you type "logger." add 7/10 times get what you want, sometimes even better), incorporating variables from the context around. This speeds up the tedious parts of programming when you are finalizing the code (adding docs + tracing).

Honestly, Copilot saves me so much time every week while turning chores into a really fun time.

Re:Kite is saying farewell, and is open-sourcing all of its ... by trip-zip on Sunday November 20, 2022 @06:22PM (#33686634)
I honestly thought I'd never use copilot, but when I need to write something to interface with XML via a SOAP API, boy copilot is my best friend...
Re:Kite is saying farewell, and is open-sourcing all of its ... by JustLurking2022 on Sunday November 20, 2022 @06:23PM (#33686643)
That code is wretched... Why have serializer logic embedded in a data object, especially when .NET provides generic discrete serializers?
Re:Kite is saying farewell, and is open-sourcing all of its ... by jackcviers3 on Monday November 21, 2022 @02:50AM (#33689974)
Yeah, tabnine kills it at the data entry parts of test dev as well for sure.
Re:Kite is saying farewell, and is open-sourcing all of its ... by mattwad on Sunday November 20, 2022 @05:16PM (#33686033)
The best part about it for me is just the Intellisense (in Typescript). I'm using it on probably 3/5 lines that I write as a smarter version, but I rarely use it to do more than finish the current line I am writing.
Re:Kite is saying farewell, and is open-sourcing all of its ... by satvikpendem on Sunday November 20, 2022 @05:09PM (#33685961)
Yes, if you show an example, or even have the test file open, it will make the other tests for you.
Re:Kite is saying farewell, and is open-sourcing all of its ... by dboreham on Sunday November 20, 2022 @05:31PM (#33686180)
I wonder if this says something about the nature of test code?
Re:Kite is saying farewell, and is open-sourcing all of its ... by djbusby on Sunday November 20, 2022 @05:35PM (#33686219)
Sure, it's been loads of boilerplate since forever.
Re:Kite is saying farewell, and is open-sourcing all of its ... by throwawaysleep on Sunday November 20, 2022 @06:14PM (#33686570)
Tests often have tons and tons of boilerplate
Re:Kite is saying farewell, and is open-sourcing all of its ... by jacurtis on Sunday November 20, 2022 @07:18PM (#33687101)
Test are always mostly boilerplate and rarely include anything crafty.

95% of tests are: instantiating a class, running a method, and then asserting that the result. Tests do not or should not be crafty creative code snippets. They are boring functional code blocks by design and most are very similar, only changing out inputs and assertions between tests.

Re:Kite is saying farewell, and is open-sourcing all of its ... by paledot on Sunday November 20, 2022 @07:43PM (#33687287)
I'd go so far as to say if your test is doing something crafty, you're doing tests wrong. Maybe in a mock or fixture, but that's a write-once sort of affair.

I also don't apply DRY (don't repeat yourself) to tests. Tests should be independently readable beginning to end, no context needed. After all, the true value of a unit test is to take a block of code too complicated to easily fit in your mind, and break it down into a series of examples simple enough to fit.

Re:Kite is saying farewell, and is open-sourcing all of its ... by simonw on Sunday November 20, 2022 @06:13PM (#33686561)
I wrote up some notes on a recent experience I had writing tests with Copilot here: https://til.simonwillison.net/gpt3/writing-test-with-copilot

Once you get the hang of how to prompt it (mainly through clever use of comments) it can be a HUGE time saver.

Re:Kite is saying farewell, and is open-sourcing all of its ... by morelisp on Sunday November 20, 2022 @05:27PM (#33686141)
How are you validating the quality of its tests? Are you trying any mutations, checking branch coverage, etc.?
Re:Kite is saying farewell, and is open-sourcing all of its ... by simsla on Sunday November 20, 2022 @07:06PM (#33687010)
I'd assume you still read the generated code, as if you're reviewing a PR
Re:Kite is saying farewell, and is open-sourcing all of its ... by jrsj on Sunday November 20, 2022 @05:45PM (#33686333)
Kite has been around for a lot longer, if anything Copilot was Github copying them
Re:Kite is saying farewell, and is open-sourcing all of its ... by esperent on Sunday November 20, 2022 @05:54PM (#33686408)
I don't think it's reasonable to say either was copying. AI assisted tooling is obvious and people have been waiting decades for the tech to reach a point where they can build these tools. Kite tried to get in early - too early probably - but even if they were the very first they didn't invent the idea.
Re:Kite is saying farewell, and is open-sourcing all of its ... by forgotpwd16 on Sunday November 20, 2022 @08:00PM (#33687416)
Interestingly checking previous submissions going back to 2016 the project had subtitle "programming copilot."
Re:Kite is saying farewell, and is open-sourcing all of its ... by lolinder on Sunday November 20, 2022 @05:54PM (#33686402)
> Just the time saved writing tests alone pays for it.

This, so much. My code since using Copilot is easily ten times better tested than it was before, and I wasn't especially lazy when it comes to testing.

Given 1-2 hand-written unit tests, Copilot can start filling in test bodies that correctly test what's described in the function name. When I can't think of any more edge cases, I'll go prompt it with one more @Test annotation (or equivalent in another language) and it will frequently come up with edge cases that I didn't even think of and write a test that tests that edge case.

(One great part about this use case for those who are a little antsy about the copyright question is that you can be pretty darn confident that you're not running a risk of accidental copyright violation. I write the actual business logic by hand, which means copilot is generating tests that only interact with an API that I wrote.)

Re:Kite is saying farewell, and is open-sourcing all of its ... by Sn0wCoder on Sunday November 20, 2022 @07:14PM (#33687065)
Super interested in how to prompt copilot for testing. I pay for it myself as high paid developer and love the boilerplate autocomplete. Just joined a new team and my lead is a good person, but rookie TS dev (as is the whole team). They are forcing me to adapt their coding style and copilot seems to see the code in the repo and emulate it. Saves me so much time and just got a 30% pay raise to join this team so if they want it wrong copilot helps me code it wrong. Also does it correctly when in my own repos so best of both worlds. If someone writes up a blog about copilot and testing would be super appreciated. Long story short I joined the team at the end of the release and NO unit tests and no linting. I turned on the linter with a git hook to only lint the files staged for checkin. Got some pushback but eventually made it though code review. After the release they want me to write all the unit/integration tests and I am already planning to automate most of it, but any advice / suggestions are welcomed. Before anyone says leave the company not an option! Just signed 2 year commitment and I am a person of my word. Plus I like my team just not the code base. If anyone is in a similar boat and wants to collaborate on a solution please leave a comment here. I will update my HN profile with a burner email that you can contact me for my real email. Or I can contact you though your profile. Once it’s done might have a product who knows…. Hopefully no one from my team sees this post cause I am going to tell them they are all hand coded and working my ass off.
Re:Kite is saying farewell, and is open-sourcing all of its ... by madsbuch on Monday November 21, 2022 @06:52AM (#33691557)
APIs are not copyrightable (Oracle vs Google). However, the code that interacts with an API might be.

Regardless, it is interesting to think about what domains are easier to generate effective models for. I would expect it to be easier to generate a supervised model <test description> => <test code>. My intuition is also, that it is easier to generate React component code, and harder to generate feature code.

Re:Kite is saying farewell, and is open-sourcing all of its ... by qw3rty01 on Tuesday November 22, 2022 @01:07AM (#33702594)
Unfortunately that ruling was overturned and APIs were found to be copyrightable: https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_...
Re:Kite is saying farewell, and is open-sourcing all of its ... by UncleEntity on Tuesday November 22, 2022 @03:11PM (#33710771)
Which was later overturned by the Supreme Court if you read farther down the article you posted the link for.

And was all over the tech news at the time.

I kind of suspect you are intentionally trolling…

Re:Kite is saying farewell, and is open-sourcing all of its ... by qw3rty01 on Tuesday November 22, 2022 @07:15PM (#33713379)
Incorrect, the case was appealed to the supreme court and the appeal was denied, so the lower court ruling held.

What was ruled by the supreme court was that Google's usage of the API (which had already determined to be copyrighted) fell under fair use in copyright law.

Re:Kite is saying farewell, and is open-sourcing all of its ... by dragonwriter on Tuesday November 22, 2022 @07:26PM (#33713474)
> Incorrect, the case was appealed to the supreme court and the appeal was denied, so the lower court ruling held.

Kind of; the appeal denied was an interlocutory appeal (an appeal before final judgement), so the lower court ruling was left in place until final resolution of the case potentially to be settled in any final appeal.

However, while copyrightability of APIs was raised on the final appeal, the Supreme Court sidestepped it, ruling that because Google’s use was fair use even if the API was copyrightable, it was unnecessary to decide the copyrightability question at all. So the Federal Circuit decision remains in place on copyrightability.

On the gripping hand, though, that decision really doesn't matter much because Federal Circuit decisions on issues outside of those it has unique appellate responsibility aren’t binding precedent on trial courts (it is supposed to apply the case law of the geographic circuit that would otherwise apply, but its rulings don’t have the precedential effect that rulings of that circuit would have.)

So, basically, as far as appellate case law, Oracle v. Google provides no binding precedent on copyrightability of APIs, but precedent that at least one pattern of API copying is fair use if APIs are copyrightable.

Which isn’t encouraging for anyone looking to protect an API with copyrights.

Re:Kite is saying farewell, and is open-sourcing all of its ... by matkoniecz on Monday November 21, 2022 @07:45AM (#33691914)
> which means copilot is generating tests that only interact with an API that I wrote

It bases this generated test cases on other similar test cases in other software, including GPL licensed

Re:Kite is saying farewell, and is open-sourcing all of its ... by UncleEntity on Tuesday November 22, 2022 @03:13PM (#33710814)
Which doesn’t matter unless you are distributing said test cases.
Re:Kite is saying farewell, and is open-sourcing all of its ... by bachmitre on Sunday November 20, 2022 @07:50PM (#33687351)
I second that
Re:Kite is saying farewell, and is open-sourcing all of its ... by vjust on Monday November 21, 2022 @01:43AM (#33689586)
I like CoPilot and paid for it out of pocket. I think its worth it. Its sometimes like having a smart programmer pairing with you.
Re:Kite is saying farewell, and is open-sourcing all of its ... by omnicognate on Monday November 21, 2022 @03:00AM (#33690032)
> If my company didn't pay for it, I would

Testimonials of this form are near worthless to a company. Maybe it's true for you. Statistically, it's highly likely to be misleading.

People overestimate their willingness to pay for something for a number of reasons, but one of the biggest is that they incorrectly visualise what the choice to pay or not looks like. They often imagine a moment of abstract choice after which everything remains exactly the same but some small amount of money magically vanishes from their bank account. In reality, paying for something is a tedious inconvenience, and not paying for it more often takes the form of never getting round to putting your card details in than consciously deciding "this isn't worth it".

It can be taken to questionable extremes, but there's truth in the idea that the only real evidence as to what customers will do is what they actually do, not what they say they will do. I don't know if their interpretation is correct, but it sounds like Kite at least has evidence of the former sort.

Re:Kite is saying farewell, and is open-sourcing all of its ... by eloff on Tuesday November 22, 2022 @10:07AM (#33706572)
It's my personal anecdote, I make no claims that other people feel the same way. If they were logical, they should though. Paying $10 to get several hundred dollars in productivity gains is a no-brainer. It's the same reason I pay for a second monitor, a properly comfortable chair and desk, a top end computer, and JetBrains. At software engineer salaries, even small productivity improvements pay off handsomely.
Re:Kite is saying farewell, and is open-sourcing all of its ... by mrtranscendence on Tuesday November 22, 2022 @10:33AM (#33706851)
I don't know about software engineering more generally, but I found it worse than useless for my work in data science (machine learning and ETL pipelines), spitting out code that was so wrong that it couldn't be salvaged. I suspect there's a wide variance in the degree to which Copilot will indeed pay for itself.
Re:Kite is saying farewell, and is open-sourcing all of its ... by bastardoperator on Monday November 21, 2022 @11:47AM (#33694665)
^ This, I may not use copilot as much on production code, but the testing code it produces makes it easily worth it from a time saved and coverage perspective.