I use this sort of stuff extensively when doing mathematical computing and statistics because there's usually a clear mathematical boundary. Once you're inside it, it's relatively easy to write down global properties (theorems) of your code's API.
The moment you cross that boundary your testing apparatuses have to get more complex and your tested properties less well-defined. Unit tests are hazier than quickcheck properties and integration tests hazier still.
This continuum seems to be precisely the same as the code reuse continuum. Highly abstracted, testable code with a shapely API is a highly reusable library whether you like it or not. Maybe it's being called by other code, maybe it's being called by your UI, maybe it's being called by the user themselves.
If a random testing framework is available in your language they really should be integrated as they are able to come up with some pathological examples.
My integration tests serve two purposes:
1) Runs the code in an repeatable and isolated environment to verify I didn't do anything stupid like misname a variable, or treat something nil as an object.
2) Validate my unique application logic.
I don't think #2 goes away with other languages, but #1 changes dramatically. I've written Ruby for years, so this isn't an "outsider looking in" opinion.
Whether conscious or not, I believe move Rubyists gravitate to testing because of both #1 and #2.
#2 can be satisfied with unit tests, but in my experience, the suite becomes a lot more flexible when validating it in terms of integration tests.
The basic thing about tests is that once you have them passing, they represent statements about constraints on the program. In other words, they express your opinion of things that should not change. Unit tests are a bet that certain aspects of the implementation will not change. Integration tests are a bet that certain aspects of the externally visible behaviour will not change.
Libraries tend to be smaller and with well-defined responsibility. Applications tend to be bigger and have many responsibilities. In general, I think it’s true that the requirements for libraries change less often than the requirements for applications. I think this leads us to expect that applications may need to be rewired “under the hood” and have their implementations changed as responsibilities are added, removed, or changed.
This, I believe, leads us to want to unit test applications less, because a unit test expresses implementation semantics, and we expect application implementations to change. No what about integration tests? Well, if we’re unit testing less in the application, we need to make up for it by integration testing more, otherwise where do we get our confidence?
Now if we throw the words “library” and “application” away, this suggests to me that those parts of the code that are small and tight and with a single, clear responsibility should be unit tested, while those parts that involve a lot of what the AOP people call ‘scattering and tangling,’ should be integration tested.
Thoughts?