News for hackers. Stuff that matters.
Re:GitHub Copilot by cbsks on Tuesday June 29, 2021 @01:07PM (#27678728)
I don't think we need to start looking for new career paths yet. This example has a few bugs and it took me longer to track them down than it would have to write it myself:

  #!/bin/bash
  # List all python source files which are more than 1KB and contain the word "copilot".
  find . \
    -name "*.py" \
    -size +1000 \
    -exec grep -n copilot {}\;
"-exec grep -n copilot {}\;" needs to have a space before the semicolon otherwise find fails with "find: missing argument to '-exec'".

The "1000" in "-size +1000" has a unit of 512 byte blocks so it is looking for files that are greater than 512000 bytes, not 1KB. This would be very easy to miss in a code review and is one of those terrible bugs that causes the code to mostly work, but not quite.

https://linux.die.net/man/1/find

Re:GitHub Copilot by nemetroid on Tuesday June 29, 2021 @05:27PM (#27681939)
It also doesn't list the files. It prints all matching lines (and their line numbers), without the corresponding filenames.
Re:GitHub Copilot by cbsks on Tuesday June 29, 2021 @09:00PM (#27683732)
You are right! I missed that. The number of bugs to lines of code ratio is approaching one.
Re:GitHub Copilot by thangalin on Wednesday June 30, 2021 @01:35AM (#27685260)
The following line is quite widespread in use, but not as portable as it could be:

    #!/bin/bash
For increased portability, respect users' PATH environment variable using:

    #!/usr/bin/env bash
Using #!/bin/bash could lead to subtle bugs on systems with more than one version of bash installed, or outright breakage on systems where bash isn't installed at /bin/bash. (OpenBSD is known to use /usr/local/bin/bash, for example.)
Re:GitHub Copilot by mkl on Wednesday June 30, 2021 @04:29AM (#27686288)
That isn't a Copilot bug though, it was typed by a person.
Re:GitHub Copilot by cbsks on Wednesday June 30, 2021 @09:50AM (#27688492)
I wonder if copilot would have produced better code if the user had used /usr/bin/env instead. If you already have buggy code, will copilot also suggest buggy code?
Re:GitHub Copilot by mitjak on Tuesday June 29, 2021 @04:13PM (#27681131)
the main argument against Copilot for me. it takes longer to grok existing code than just write it from ground up.
Re:GitHub Copilot by toxik on Tuesday June 29, 2021 @05:15PM (#27681821)
This is actually a pretty important thing to understand. It can be better to rewrite than fix an existing mess. It’s similar to construction work in that sense: if you rebuild, you know what’s inside the walls.
Re:GitHub Copilot by larkinrichards on Tuesday June 29, 2021 @09:36PM (#27683927)
If your house doesn’t have fire proofing (Sheetrock / lathe & plaster) then you know what’s inside the walls.
Re:GitHub Copilot by toxik on Wednesday June 30, 2021 @03:40AM (#27686001)
Kind of comical to open a rebuttal with “if”, showing you don’t in fact know. That aside - no, you don’t know necessarily.
Re:GitHub Copilot by ok2938 on Tuesday June 29, 2021 @05:49PM (#27682158)
That's why this copilot won't fly. The junior programmer will not be able to spot subtle errors but will kind-of feel "productive" by some random pastes from a giant brain, which cannot be interrogated.

If anything, I see copilot generating more work for existing, senior programmers - so there you have it.

Re:GitHub Copilot by kbenson on Tuesday June 29, 2021 @09:09PM (#27683790)
Is that your main argument against Stack Overflow? Because for some percentage of people the use cases will be similar (that is, learning some quick ways to do something that they can then explore and learn about).

Sometimes resources like these are used as references and the solution is used as is. Sometimes they are used as a survey, and it's more like asking a librarian "I want to learn more about how to X" and having them give a short exposition and point out some sources to study. It's important not to let you view of it's usefulness as one type of resource bleed into your view of how useful it might be for the other.

As a learning resource, this is very interesting.

Re:GitHub Copilot by nuccy on Tuesday June 29, 2021 @11:39PM (#27684587)
When one gets snippets from Stack Overflow, he/she doesn't assume it is tailored specifically for the task, so extra check, modification are applied. With copilot people (eventually, especially beginners probably) will assume that the code produced by "magic" AI does exactly what they asked for.
Re:GitHub Copilot by tusharsadhwani on Wednesday June 30, 2021 @02:54AM (#27685699)
Funnily this problem seems trivial when you have good test coverage for your code. Especially trivial compared to the insane amount of help you could get in writing boilerplate and simple logic from a magical tool like this, if it delivers on its promises.
Re:GitHub Copilot by bogwog on Tuesday June 29, 2021 @08:23PM (#27683525)
Plus if you use this for anything non-trivial, it's functionally similar to using a third-party dependency, except the licensing terms are vague and there's no one to ask for help.

I could see it being useful maybe for testing. Even if it straight up copies code from the Linux kernel, you don't usually ship tests to customers so the GPL (probably) isn't a problem.

But the problem with testing is that you need to be 100% sure about how it behaves, and an AI generated test might not be reliable enough to be useful.

Re:GitHub Copilot by mitjak on Wednesday June 30, 2021 @05:31PM (#27693609)
i'd argue the danger is even greater with tests bc the entire point of tests is to provide certainty. wrong tests are a false sense of safety that's unquantifiable without understanding every single line.
Re:GitHub Copilot by awestroke on Tuesday June 29, 2021 @04:04PM (#27681016)
I read this as a criticism of bash
Re:GitHub Copilot by hashingroll on Tuesday June 29, 2021 @04:23PM (#27681244)
I don't see why. It is on Copilot to produce syntactically correct code that at least doesn't fail to run, even if we ignore the correctness.
Re:GitHub Copilot by cbsks on Tuesday June 29, 2021 @08:56PM (#27683709)
This doesn’t really have anything to do with bash, just the ‘find’ posix utility. But I agree that the arguments not intuitive.
Re:GitHub Copilot by EGreg on Wednesday June 30, 2021 @01:30AM (#27685236)
Give them a year. It sounds like the things people said about chess engines that played stupidly

Just kidding... writing imperative code is fundamentally different than most AI recognition tasks, ie you can have GPT-4 produce consistently nice HTML but not C++

Re:GitHub Copilot by arp242 on Wednesday June 30, 2021 @06:41AM (#27686986)
Also: "-exec grep {} \+" will be a lot faster if you have many files, and you will get the actual filename too instead of just the line number which is what the comment says it should do.
Re:GitHub Copilot by fomine3 on Tuesday June 29, 2021 @10:55PM (#27684370)
I convinced I'll do same mistake.
Re:GitHub Copilot by lostmsu on Wednesday June 30, 2021 @12:54AM (#27685010)
I am not surprised. Bash is notoriously unreadable.
Re:GitHub Copilot by mkl on Wednesday June 30, 2021 @04:33AM (#27686313)
There is no bash code in that script, it's POSIX find.
Re:GitHub Copilot by Tyr42 on Wednesday June 30, 2021 @10:34AM (#27689045)
POSIX find is also terrible. I'm sorry but I go out of my way to avoid using it as it's always a pain.