Re:Unit testing a TCP stack
by jsnell on Thursday July 09, 2015 @04:26PM
(#9860444)
I watched the Usenix talk on packetdrill, and while it's a really neat tool it seems to me that it doesn't really support timing very well.
The tests are run in real time - running 650 tests take almost half hour, which would normally be considered completely unacceptable for any kind of TDD. The timing can also vary from one run to another, so all timestamps have a few milliseconds of wiggle room automatically, and a small proportion of runs (1/2500) still fail randomly due to timer inaccuracies. And unit tests that fail non-deterministically are really unpalatable. Is the talk outdated re: these issues?
Great hack despite that, and I would certainly have loved to write a scripting language for expressing tests rather than use C++. And the packet syntax they've chosen is probably superior to what I ended up with. But what I describe in the post dates back to 2011, so packetdrill wasn't yet around for us to take inspiration from :-/
The tests are run in real time - running 650 tests take almost half hour, which would normally be considered completely unacceptable for any kind of TDD. The timing can also vary from one run to another, so all timestamps have a few milliseconds of wiggle room automatically, and a small proportion of runs (1/2500) still fail randomly due to timer inaccuracies. And unit tests that fail non-deterministically are really unpalatable. Is the talk outdated re: these issues?
Great hack despite that, and I would certainly have loved to write a scripting language for expressing tests rather than use C++. And the packet syntax they've chosen is probably superior to what I ended up with. But what I describe in the post dates back to 2011, so packetdrill wasn't yet around for us to take inspiration from :-/