Re:Stop Using Mocks
by aarongough on Wednesday March 23, 2011 @01:06PM
(#2360251)
That's generally referred to as 'mutation testing'. There are a number of systems that enable this in dynamic languages, the one I am most familiar with is Heckle: https://github.com/ryansobol/heckle
Mutation testing is great, it highlights all the areas where your tests aren't written well, and teaches you to write better tests. I'm not sure that using it all the time is the best use of resources in most cases though...
Here's how it works:
for each statement in your code, break it, then run your entire test suite. If none of your tests have uncovered a problem, mark the code as uncovered. Repeat for each code statement.
It's crazily expensive to compute, but it's so much more accurate than to say that a line of code was hit or not. It measures whether the code had a functional impact on the overall behavior.