Re:The maintainability of unit tests
by jerf on Thursday February 25, 2010 @11:52AM
(#1150972)
"How do you deal with things that write files out or communicate with other processes?"
I have the tested code write the files out, then I have the test code examine the files. I have the tested code communicate with the other processes, then examine the results, possibly querying the other processes to see whether they got what they should get and did what they should do. I make it so it's easy to put the files in a temporary location, easy to start up test processes for the test to communicate with, and then test the system at this higher level.
I am aware that this is counter to the unit testing dogma. I am aware this requires certain amounts of access to the code of all parts of the system, but I usually either have it or make it a priority to get it. So far, the predictions of doom if I do this have failed to manifest.
And I've learned that when someone predicts doom that fails to manifest, that's usually because the predictions are based on untested theories and not actual experience. This happens to an absolutely shocking degree in our industry; the amount of software design dogma that is basically just one guy with a PhD's opinion he stated in a paper once 20 years ago, followed by a huge game of telephone, can hardly be understated.
(Your homework exercise, if you don't believe this, is to find the published, peer-reviewed paper in which it was determined that unit tests work best if they only test the smallest possible units. Good luck. If that doesn't float your boat, pick your favorite bit of OO-design dogma, like the Law of Demeter, and trace back where it came from. Even good ideas generally don't have the sort of actual support you might think from how far the ideas tend to spread. My take-away from that is that not only is it OK to question, it is a necessity.)
Everywhere your test code tests something other than the actual code is a place where bugs can hide. Mock objects are basically a systematic way for providing those places for bugs to hide. I agree with silentbicycle that they have a small place, but IMHO systematic use of them is wrong.
I have the tested code write the files out, then I have the test code examine the files. I have the tested code communicate with the other processes, then examine the results, possibly querying the other processes to see whether they got what they should get and did what they should do. I make it so it's easy to put the files in a temporary location, easy to start up test processes for the test to communicate with, and then test the system at this higher level.
I am aware that this is counter to the unit testing dogma. I am aware this requires certain amounts of access to the code of all parts of the system, but I usually either have it or make it a priority to get it. So far, the predictions of doom if I do this have failed to manifest.
And I've learned that when someone predicts doom that fails to manifest, that's usually because the predictions are based on untested theories and not actual experience. This happens to an absolutely shocking degree in our industry; the amount of software design dogma that is basically just one guy with a PhD's opinion he stated in a paper once 20 years ago, followed by a huge game of telephone, can hardly be understated.
(Your homework exercise, if you don't believe this, is to find the published, peer-reviewed paper in which it was determined that unit tests work best if they only test the smallest possible units. Good luck. If that doesn't float your boat, pick your favorite bit of OO-design dogma, like the Law of Demeter, and trace back where it came from. Even good ideas generally don't have the sort of actual support you might think from how far the ideas tend to spread. My take-away from that is that not only is it OK to question, it is a necessity.)
Everywhere your test code tests something other than the actual code is a place where bugs can hide. Mock objects are basically a systematic way for providing those places for bugs to hide. I agree with silentbicycle that they have a small place, but IMHO systematic use of them is wrong.