News for hackers. Stuff that matters.
Re:The sad state of test frameworks by vortico on Tuesday March 21, 2017 @01:47AM (#13920127)
For C and C++, what's wrong with using assert() in an integration_test() function or many <func>_test() functions, called when passed a --test flag or compiled as a separate binary?

For me, benchmarking isn't really useful unless it's a profiler in a real world use situation. Otherwise you have no idea how often each API function is called.

Re:The sad state of test frameworks by koja86 on Tuesday March 21, 2017 @05:36AM (#13920823)
For C++ I find both Google Test and Boost Unit Test Framework pretty usable. They both have some quirks but IMHO no show-stoppers. I prefer these to custom solutions because don't want to maintain yet another tool but to focus on my problem domain and it is nice when some fellow developers actually have experience with these tools.
Re:The sad state of test frameworks by codr4life on Tuesday March 21, 2017 @10:37AM (#13922513)
The posted approach clocks in at 300 lines of C, and comes with the freedom to add whatever features you want. Testing is as fundamental as it gets; and unless you're aiming to write the One True Testing Framework, it's at most a days work to get one up and running. Google Test and Boost Unit try to hard to be everything for everyone, which makes them too complex; while failing to add anything that was missing from plain functions and asserts.
Re:The sad state of test frameworks by gpderetta on Tuesday March 21, 2017 @08:09AM (#13921442)
I prefer Boost.UnitTest to Google Test, which I also like. Boost.UnitTest has very nice sugar, but the big problem is the very slow compilation times.
Re:The sad state of test frameworks by imron on Tuesday March 21, 2017 @08:56AM (#13921727)
That's why I like catch [0]

Super easy to use, header only so no need to compile anything beforehand, no external dependencies, reasonably fast compilation time, and does everything I need.

0: https://github.com/philsquared/Catch

Re:The sad state of test frameworks by to3m on Tuesday March 21, 2017 @09:47AM (#13922116)
Funny... I dumped Catch after a couple of days due to the terrible build times ;)

The fact it took over my programs' command line options and wouldn't give 'em back didn't help either. (Many of my tests are data-driven and use command line options to indicate where to read the data from - I already have a library to handle this! I didn't see - and still don't - why a test framework should be getting involved.)

Re:The sad state of test frameworks by braveo on Tuesday March 21, 2017 @02:17PM (#13924722)
I actually tried out Catch last night and dumped it almost immediately for the exact same reason. I was blown away by how slow it was compared to the rest of my (admittedly small) program in terms of build times.

I found using boost was a smaller hit to build times than Catch, and the suggestion of building catch in a separate translation unit didn't really help.

I'm guessing if your build times are already godawful you may not notice, but I sure as shit did.

Ultimate I found and went with dessert: https://github.com/r-lyeh/dessert

It's stupidly simple and doesn't do a lot of things that other C++ testing frameworks due, but it builds lightning quick and does 2 things I expect out of testing frameworks.

1. it tests, and 2. it reports failures.

I'll probably run into severe limitations that I can't deal with/work around, but for now I'm pretty happy with it and even if I end up using something else in 6 months, I'll still consider it worthwhile.

Re:The sad state of test frameworks by thedonkeycometh on Tuesday March 21, 2017 @06:17AM (#13920952)
I think thats the main rub, think about the maintainers. Using a bespoke test framework, is often a way of introducing conflict, no matter how good a coder you are. As Jeff Atwood says, imagine the maintainer being a psychopath.. https://blog.codinghorror.com/coding-for-violent-psychopaths...
Re:The sad state of test frameworks by codr4life on Tuesday March 21, 2017 @10:41AM (#13922550)
Would you rather have your psychopath stalkers tearing their hair out over the Boost dependency you dragged in? Once written it stays mostly the same, there is not much to maintain. I can't see why rolling your own framework would introduce conflicts; unless you're in an environment where writing code to solve problems is frowned upon, but then you have worse problems.
Re:The sad state of test frameworks by pg314 on Tuesday March 21, 2017 @02:26AM (#13920265)
Nothing. That's the way I usually test my C code.

It has the drawback that one failing test aborts the whole program. The error messages are not the most informative, but it does the job.

Re:The sad state of test frameworks by vortico on Tuesday March 21, 2017 @07:04AM (#13921166)
Although hacky,

    assert(test_foo() /* Checks the sanity of Foo */);
does the job for the error messages, and replacing abort() with a custom function like `void check(int condition, const char *format, ...);` can make it not abort the program on an error.

I prefer the regular assert() though because I want the build to fail by an aborting test program, and all I care is the line number of failure. Another benefit is that abort() will pause gdb in the exact place that it crashed, with the stack intact if you want to inspect it.

Re:The sad state of test frameworks by cormacrelf on Tuesday March 21, 2017 @08:05AM (#13921432)
You can also use the comma operator to document your assert()s, if you don't like using comments.

    assert(("test_foo() should be true", test_foo()));
https://en.wikipedia.org/wiki/Comma_operator#Asserts
Re:The sad state of test frameworks by gpderetta on Tuesday March 21, 2017 @08:10AM (#13921449)
I have used often and also seen in the wild the following form:

   assert(condition && "message");
Re:The sad state of test frameworks by klodolph on Tuesday March 21, 2017 @04:00AM (#13920517)
This is easy enough to solve on Unix... simply fork() before each test. Just make sure that the master is single-threaded or this won't work.
Re:The sad state of test frameworks by gpderetta on Tuesday March 21, 2017 @08:11AM (#13921453)
Also an option is to siglongjmp out of the abort signal handler.
Re:The sad state of test frameworks by codr4life on Tuesday March 21, 2017 @10:44AM (#13922585)
I've found it worthwhile to add a thin facade on top of assert to support out-of-band reporting of errors even if NDEBUG is disabled: https://github.com/codr4life/libc4/blob/master/src/c4/assert...
Re:The sad state of test frameworks by codr4life on Tuesday March 21, 2017 @10:29AM (#13922454)
There's nothing wrong with assert, that's what I use for checking conditions as well. The framework posted is just a thin facade on top of regular functions and asserts to provide fixtures, dynamic grouping and benchmarks. Knowing how long a specific set of tests take to run for N repetitions is useful information. Since I wrote the test, I know exactly how many times the tested API is called. This is about spotting problems and trends early, once I find something interesting I'll drop down to profiling.