Re:GitHub Copilot X: The AI-powered developer experience
by jchw on Wednesday March 22, 2023 @10:50AM
(#35261913)
Not very impressed with either Copilot or ChatGPT for programming purposes. I'm sure it will be interesting for some people, but whenever I really feel like I could benefit from an intelligent AI, it usually falls flat. Most commonly, it will just generate calls that don't exist, and if you try to get it to implement them, it will write bullshit.
Now the thing is, maybe it really does improve developer productivity by virtue of letting the developer autocomplete boring things. To be fair, I did see some opportunity here. But on the other hand, it also seemed to harm productivity, as I had to carefully review everything it did, and eventually had to disable Copilot so that it would stop annoying me with nonsense completions every time I stopped typing.
I'm not really sure if language models are actually getting closer to what we really want, or if they're getting closer to some other maxima that just looks similar. Clearly they're doing something that is novel and probably even useful, but with increasing parameter counts it seems that the underlying problems are not changing. Something certainly seems like it must be fundamentally wrong...
[ Parent
| on HN ]
Re:GitHub Copilot X: The AI-powered developer experience
by teaearlgraycold on Wednesday March 22, 2023 @11:21AM
(#35262409)
I use them all the time with success. For copilot it’s very useful for writing tests and autocompleting repetitive code. Like writing a Typescript interface that matches a JSON object, or listing properties of an object into an array. ChatGPT is at worst a good rubber duck, and at best a superpowered documentation tool. If I ever get confused by docker or some popular JavaScript library it’s able to give me a custom tailored answer that usually solves my problem on the first or second try.
Neither of them are any good at just writing a ton of code (unless the code is mostly boilerplate). But they’re amazing keystroke savers and a next generation stack overflow.
Re:GitHub Copilot X: The AI-powered developer experience
by danenania on Wednesday March 22, 2023 @01:07PM
(#35264030)
What would be amazing for Copilot + Typescript is if it could somehow integrate with the TS compiler (or perhaps just get sufficient fine-tuning on the TS type system) so that it only generates valid/type-safe suggestions.
Re:GitHub Copilot X: The AI-powered developer experience
by FractalHQ on Thursday March 23, 2023 @07:13AM
(#35272930)
I think about this often. If copilot could hook into the Typescript language server, it would be much less likely to hallucinate and generate blatantly wrong suggestions.
Re:GitHub Copilot X: The AI-powered developer experience
by 8organicbits on Wednesday March 22, 2023 @07:55PM
(#35268731)
How often are you seeing bogus documentation? I see that concern often mentioned. I tried ChatGPT and the first question got an obviously wrong answer. That seems worse than stackoverflow to me. Maybe I'm not holding it right?
Re:GitHub Copilot X: The AI-powered developer experience
by teaearlgraycold on Thursday March 23, 2023 @03:12AM
(#35271410)
I think it entirely depends on how popular the library is. React for example is so common online that GPT has the whole API figured out.
Even if it’s wrong the stakes are low. Just paste the error back into the chat and it will give you an alternative.
But the real solution here is the new Copilot X for docs. Feed in the code base and all documentation and now you have the real source of truth at the helm.
Re:GitHub Copilot X: The AI-powered developer experience
by thebackup on Wednesday March 22, 2023 @10:57AM
(#35262037)
I second this. My impressions so far are pretty much the same.
Have been trying to get ChatGPT to generate code for some algorithms that are pretty close to well known implementations but with some twists and it absolutely fails no matter how much I tried to provide hints to change the suggestions it gave me. In the end I just went back to hand coding them since it actually saves me the frustration of trying to get ChatGPT to bend in the direction I need.
Re:GitHub Copilot X: The AI-powered developer experience
by Donald on Wednesday March 22, 2023 @02:31PM
(#35265217)
Have you tried GPT-4?
Re:GitHub Copilot X: The AI-powered developer experience
by thebackup on Wednesday March 22, 2023 @03:32PM
(#35266038)
Will give it a go.
But probably it’ll take another generation or two before it’s good enough.