News for hackers. Stuff that matters.
Re:Ask HN: P=NP, what do you do? by praptak on Tuesday June 02, 2009 @04:25AM (#637521)
This would probably produce a program that cheats a lot. Perhaps not by simply hardcoding the answers to the tests, but by finding a good compressing function for the set of (input, output) pairs in the test. I bet this would seldom provide the expected answers on inputs not covered by the tests.
Re:Ask HN: P=NP, what do you do? by wlievens on Tuesday June 02, 2009 @07:29AM (#637636)
I second this. Overfitting can be a problem even in human-written test scenario's. Usually overconfidence in your test suite is a cause.