News for hackers. Stuff that matters.
Re:Testing is waste of time, I know that my code works by ssp on Friday December 24, 2010 @12:16PM (#2037578)
Test suites can make you less careful because you start relying on the tests flagging any bugs. And then when the test suite does find a bug, you just hack the code until the test suite passes instead of fixing the underlying problem. This is the same problem as using microbenchmarks for performance work: you end up tuning to the benchmark instead of to the real world.
Re:Testing is waste of time, I know that my code works by jerf on Friday December 24, 2010 @12:32PM (#2037612)
By "you", I think you may mean "I". If ssp is hacking the test suite to bypass a failing test, ssp has the problem, not the test suite. jerf does not have much trouble with that, jerf has experienced "the one failing test that turns out to reveal a major underlying problem followed by a real fix that it would have taken him multiple customer-losing hard-to-reproduce bugs in field to learn about" multiple times.

Not that this is perfect, I've got just such a bug out there right now that simply refuses to be reproduced by anyone once a developer is looking at it, but I sure have far fewer of those than I would without the tests.

Re:Testing is waste of time, I know that my code works by ssp on Friday December 24, 2010 @01:25PM (#2037733)
If ssp is hacking the test suite to bypass a failing test

I'm talking about hacking the application to pass the test suite, not hacking the test suite. You can often "fix" a failing test by doing

     if (condition that failed)
           whack the application state so that the test suite will pass.
without understanding what the actual bug were. And it's not always obvious to yourself that this is what you are doing.

ssp has the problem, not the test suite

Tuning to benchmarks is not some unique character flaw of mine. When you measure some aspect of people's behavior, they will optimize to that measurement. If the measurement is a boolean PASS/FAIL from the test suite, then they will optimize their behavior to get a PASS.

But the actual desired outcome is not PASS, it is "bug free program".

Re:Testing is waste of time, I know that my code works by jerf on Friday December 24, 2010 @05:28PM (#2038131)
Hack the app, hack the test suite, I meant either equally. Which I think if you read my post is pretty clear that the main dichotomy is between "hack" and "real fix" and where the hack goes hardly matters.

Nevertheless, you are arguing that because people sometimes program to benchmarks, you are apparently better off without the benchmarks. I say that's nonsense. The solution is to use the benchmarks better. Are we programmers or automatons? (Or managers?) If you're going to be that defeatist about programming you're not going to be able to be a successful programmer under any circumstances, the entire field is a minefield of superficially-appealing optimization opportunities!

Re:Testing is waste of time, I know that my code works by ssp on Friday December 24, 2010 @05:30PM (#2038132)
Nevertheless, you are arguing that because people sometimes program to benchmarks, you are apparently better off without the benchmarks.

No, I am not.

Re:Testing is waste of time, I know that my code works by lutorm on Friday December 24, 2010 @02:09PM (#2037804)
The answer to the problem of optimization by proxy is not no optimization at all.