News for hackers. Stuff that matters.
Re:How we took our tests from 100 seconds to 100ms by Confusion on Tuesday July 21, 2015 @03:54PM (#9925260)
The distribution of when bugs are caught, and what bugs are caught, by different kinds of tests is very different. Units tests catch bugs while they are being written and when the units they test are being refactored. The bugs they catch are trivial, silly, 'would have been caught anyway'. Outside of those activities, they rarely catch bugs. On the other hand, integration tests more consistently catch bugs and catch bugs you didn't foresee, have to investigate before understanding.

You don't remember the last time a unit test caught a bug, because on average it is much longer ago and if it was a test for a unit you wrote this morning, you usually blame it on 'work in progress' and think you would have caught that trivial bug anyway.

I think the nature of units tests make it easy to underappreciate them.

Re:How we took our tests from 100 seconds to 100ms by Glide on Wednesday July 22, 2015 @11:33AM (#9930199)
This is too true. A team I was on started seeing way more value in integration tests and ended up writing way more integration tests than unit tests.

What ended up happening is that the code quality ended up worse because they stopped running tests very often because there were too many integration tests and they took too long to run. Sad part is, a lot of those tests would have made a lot more sense as unit tests.