Re:Why code review beats testing: evidence from decades of p...
by ZeroGravitas on Tuesday October 04, 2011 @07:17AM
(#3070340)
It seems to me that if there's no right answer, then whatever you do you can't possibly be wrong. That's pretty easy code to write (and make really fast at the same time).
I'm guessing there is a right answer, it just might be poorly specified. And automated or unit testing doesn't guarantee that even simple things are correct. It's just a trade-off of how much time you put into the automation versus the time you spend chasing bugs you could have caught with automation (which isn't all of them). I'd think in most real world code (for all the usual real world reasons) that people err on the too-little automated-testing side of things.
What do see as the problems with checking the md5sums? I'm guessing they're brittle if the answer isn't supposed to be exactly the same, but that reduces down to the same as checking a number is 5 when it might be between 4.8 and 5.2. It's not rocket science and getting it wrong doesn't invalidate all automated testing. At the very least the first time it throws an error the developer who wrote the bad test is going to learn something new about his system and you might sensibly use such cheaply written tests to locate errors when running code on a different OS (or version) or after minor cosmetic changes where you would expect the exact same result.
I'm guessing there is a right answer, it just might be poorly specified. And automated or unit testing doesn't guarantee that even simple things are correct. It's just a trade-off of how much time you put into the automation versus the time you spend chasing bugs you could have caught with automation (which isn't all of them). I'd think in most real world code (for all the usual real world reasons) that people err on the too-little automated-testing side of things.
What do see as the problems with checking the md5sums? I'm guessing they're brittle if the answer isn't supposed to be exactly the same, but that reduces down to the same as checking a number is 5 when it might be between 4.8 and 5.2. It's not rocket science and getting it wrong doesn't invalidate all automated testing. At the very least the first time it throws an error the developer who wrote the bad test is going to learn something new about his system and you might sensibly use such cheaply written tests to locate errors when running code on a different OS (or version) or after minor cosmetic changes where you would expect the exact same result.