Re:Mutation Driven Testing: When TDD Just Isn’t Good Enough
by tom_mellior on Saturday February 06, 2021 @05:32AM
(#26045634)
> Also every single of the author’s exemple is caught by an actually good testing tool like QuickCheck.
I'd be interested to see what a sufficiently strong QuickCheck specification of this problem would look like. I've used it a bit in the past, but not enough that I could reliably get it to produce all the interesting failure modes and know the expected result for each case.
Re:Mutation Driven Testing: When TDD Just Isn’t Good Enough
by twanvl on Saturday February 06, 2021 @06:40AM
(#26045902)
> I'd be interested to see what a sufficiently strong QuickCheck specification of this problem would look like.
I would write something like this in haskell:
spec :: [Integer] -> Property
spec xs =
length xs <= 2 ==> fun (intercalate "," (map show xs)) == sum xs
This captures the three requirements, but not the implicit fourth requirement that the function throws an exception for other inputs.
Re:Mutation Driven Testing: When TDD Just Isn’t Good Enough
by tom_mellior on Saturday February 06, 2021 @08:02AM
(#26046231)
Nor does this exercise the trimming of the substrings, for example. This is good for testing the happiest path, I agree. I was interested in the tedious testing of all the unhappy paths.
Re:Mutation Driven Testing: When TDD Just Isn’t Good Enough
by masklinn on Saturday February 06, 2021 @07:25AM
(#26046088)
> not the implicit fourth requirement that the function throws an exception for other inputs.
You could probably generate invalid inputs by taking a list of strings as input. Though of course at that point the property test has to reimplement half the function.
That's an issue I often end up having with property tests: the oracle for interesting properties is as complex as the SUT, so you end up with two of them.
Basically what this is is writing tests for your tests. And because the input of tests is functions you need to be able to generate functions. That’s nice but it’s a pain considering the only solution proposed is “just do it manually” which is neither exhaustive nor trustworthy.
Also every single of the author’s exemple is caught by an actually good testing tool like QuickCheck.
https://en.wikipedia.org/wiki/QuickCheck