Re:Why Most Unit Testing Is Waste [pdf]
by taneq on Wednesday November 01, 2017 @01:34AM
(#15598997)
Your API-spammer interface reminds me of the Macintosh "Monkey": https://www.folklore.org/StoryView.py?project=Macintosh&stor...
Yeah it's pretty similar except mine was not interesting to watch as it works :D
Re:Why Most Unit Testing Is Waste [pdf]
by josteink on Wednesday November 01, 2017 @02:16AM
(#15599145)
A test can still be simple while providing garbage edge-case data.
In fact that’s usually one of my default tests to ensure that I have a well-defined behavior even in these cases.
Yeah but the cases I had in mind, I probably coded correctly. The cases I forgot about are more likely to cause problems (weird cases in both data and especially timing / concurrency handling).
On 2 separate occasions I had an app with extensive unit tests that seemed to work fine but also seemed to have strange rare bugs.
Both times I wrote a single additional "unit" test that fired up the environment (with mocks, same as the other unit tests) but then acted like a consumer of the API and spammed the environment with random (but not nonsense) calls for several minutes. These tests were quite complex so basically the exact opposite of what you're suggesting.
Not only did I immediately find the bug, but in both cases I found like 10 bugs before I got the test to even pass for the first time.
At the same time all the small unit tests were happily passing. Because they didn't hit edge cases (both in data and in timing) that nobody had thought of.