News for hackers. Stuff that matters.
Re:How to avoid brittle code by crdoconnor on Monday March 28, 2016 @08:51AM (#11373927)
I haven't found this to be a problem at all. Integration tests follow a power law. If you segregate your tests into "smoke" and "non-smoke", run the former before merging code and the latter continuously and randomly and in parallel, you'll get notification of 95% of regressions within 5 minutes and the other 5% within a few hours.

Choosing unit tests because of speed is nearly always a premature optimization.

IME everywhere that I worked with that relied exclusively on unit tests for automated regression testing didn't trust them enough to do releases without some manual user testing, so the project fell back into waterfall mode.

Integration tests tested a lot more realistically and gave more freedom to refactor technical debt (since changing the code didn't require changing the test as often).

Re:How to avoid brittle code by Bahamut on Monday March 28, 2016 @09:48AM (#11374210)
The main benefit of unit tests is the speed one can run them at, and the feedback. Unit tests make it vastly faster to figure out the culprit in a broken sector of code almost all the time, since it narrows down what code you have to check for any logic errors. It also so happens that unit tests tend to encourage better writing of code into reuseable units.

Unit tests cannot catch everything though, and for stuff like how multiple units are wired together, that's where integration tests come into play.

A good codebase has both types - even combined they won't catch everything, but they should capture almost all business logic issues.

Re:How to avoid brittle code by crdoconnor on Monday March 28, 2016 @03:07PM (#11376538)
I've found that where code involves complex logic, unit tests often are more effective for the reasons you gave.

However, a lot (perhaps most) code is more about integrating disparate components (browsers, lots of different kinds of external APIs, databases, times, distributed task processors, etc.) rather than lots of complex logic and for that kind of code, unit tests are substantially less effective and more horrible to use whereas integration tests really shine.

I think ideally, you should keep the two kinds of code strongly segregated, with clean APIs between them, so you can easily use the appropriate kind of testing for either one.

I don't agree that a good code base necessarily has both types. A web app with relatively shallow logic and lots of integrations only really benefits from having integration tests. A library to convert human readable strings into dates likewise gains nothing from having integration tests.

Similarly, a messy spaghetti codebase with a disgusting mix of both types of code can only really benefit from integration testing until it's substantially refactored and the two types of code are segregated.

Re:How to avoid brittle code by dennisgorelik on Tuesday March 29, 2016 @12:14AM (#11379455)
> you should keep the two kinds of code strongly segregated, with clean APIs between them

Why _strongly_ segregated?

Nothing stops you to write unit tests for the methods that should be covered by unit tests and integration tests for the functionality that needs to be covered by integration tests.

How would code segregation help?

Re:How to avoid brittle code by crdoconnor on Tuesday March 29, 2016 @06:11AM (#11380501)
If the code is not well segregated you won't be able to write a clean unit test that tests the complex logic without also mocking a bunch of stuff.

That test will also often break as you refactor the various connecting APIs it surrounds.

Tests that break in the presence of changed API contracts (as opposed to bugs) are vastly less valuable than tests which only go red in the presence of a bug.

Re:How to avoid brittle code by vkou on Monday March 28, 2016 @12:06PM (#11375059)
5% regressions caught within a few hours is a huge price to pay on larger teams - especially if there are multiple other services depending on yours.
Re:How to avoid brittle code by dllthomas on Friday April 01, 2016 @11:30AM (#11405512)
A huge price to pay relative to what what alternative?
Re:How to avoid brittle code by crdoconnor on Monday March 28, 2016 @03:08PM (#11376548)
Not unless you release your code to production every 20 minutes.