Re:We need to talk about testing
by carlmr on Wednesday July 28, 2021 @10:48AM
(#27984451)
If the behavior changes, the requirements have changed or you previously had a bug. That's not brittle tests.
[ Parent
| on HN ]
Someone wants to test:
function do_add(a, b, add) {
return add(b, a);
}
And does something like: do_add(1, 2, add)
expect(add) to be called with (2, 1)
Then the function is changed to: function do_add(a, b, add) {
return add(a, b);
}
Oops, broke the test. Is this a type of behavior change that you expect would break the test? Often there are many ways to correctly do something and a good test should allow any of them.But that's not a behavioral change, only bad testing. The test wouldn't be brittle if the behavior of adding was locked down completely.
Re:We need to talk about testing
by yourenotsmart on Thursday July 29, 2021 @12:06PM
(#27998071)
Any test that uses mocks is therefore "bad testing".
Re:We need to talk about testing
by yourenotsmart on Wednesday July 28, 2021 @02:57PM
(#27988101)
The very problem is that tests failing is not necessarily due to behavior changing but implementation details not affecting behavior changing.
This is especially the case when tests rely on mocks, which don't really implement the BEHAVIOR of a dependency, but rather represent a "replay" of a specific call stack sequence.