News for hackers. Stuff that matters.
Re:Autocomplete from Stack Overflow by conjectures on Tuesday March 22, 2016 @12:02PM (#11337501)
That actually sounds realistic. A well defined test suite is probably a good target for AI. Main issue is no partial wins, which are needed for training, so you'd need to break all functionality down into absolutely minimal units.
Re:Autocomplete from Stack Overflow by chillacy on Tuesday March 22, 2016 @01:56PM (#11338475)
Also if you overfit, your code is going to look like:

  int add(int a, int b) {
    if (a == 1 && b == 1) return 2;
    if (a == -1 && b == 0) return -1;
    if (a == 13 && b == 7) return 20;
    return 0;
  }
But at least it passes my unit test suite!
Re:Autocomplete from Stack Overflow by firethief on Tuesday March 22, 2016 @02:09PM (#11338580)
Just rerun it with a lower target code complexity. You'll know you set it too low when it starts monkey-patching the test framework to avoid actually doing anything.
Re:Autocomplete from Stack Overflow by Houshalter on Wednesday March 23, 2016 @03:29AM (#11342697)
This can be avoided by optimizing for the simplest code that works. Checking many cases in multiple lines is always going to be longer and more complex than just a single operation on a single line.

Which is how AIs avoid overfitting, in general, by penalizing complexity.

Re:Autocomplete from Stack Overflow by eru on Tuesday March 22, 2016 @07:32PM (#11340824)
That's why you use property based testing like Python's Hypothesis or QuickCheck; instead of example based unit testing.