https://dlang.org/spec/unittest.html
It's been a real game changer for us in improving the quality of the code.It was pioneered in the context of logic programming and Prolog by Ehud Shapiro in his 1982 PhD thesis "Algorithmic Program Debugging". The thesis was published as an ACM Distinguished Dissertation by MIT press and is available online from:
http://cpsc.yale.edu/sites/default/files/files/tr237.pdf
Together with Leon Sterling, Ehud Shapiro later wrote a very important introductory Prolog book called "The Art of Prolog".
reverse "abcde" = "edcba"
Other, older projects include Exference[1] and Djinn[2], in decreasing order of power.Also, this is really similar to how Idris and Agda work, except they use expressive types to generate the code (using the Emacs modes) rather than test cases.
[0]: http://nautilus.cs.miyazaki-u.ac.jp/cgi-bin/MagicHaskeller.c...
> tests without cluttering the production code and excessive mocking
IMHO, tests should be an integral part of production code. In MPWTest[1], tests are typically expressed on the class side of the class under test, in a category called Testing, though it's easy to override that. This solves 2 of my major annoyances with xUnit-style testing: dual class/test hierarchies and (with static typing) the need to add public interface for the tests, which don't have privileged access.
Mocking in particular and stubbing should be eliminated as much as possible, but this is not a programming language issue[2], more a "not making architectural assumptions too early" issue.
I don't think Spec alone removes the need for mocking, using Clean Architecture is probably more important for that https://8thlight.com/blog/uncle-bob/2012/08/13/the-clean-arc...
If you structure your app so that any IO lives at the edges, it becomes possible to test all your business logic without mocking. Spec can help at that point by providing generative testing.
Since Spec
Technically, these could very well be simple commented annotations that point to a separate .test file or something, but are shown in-line by your favorite IDE or code-editor.
> I'd also like to specify various injection points for tests right as I'm writing the code
You mean, like methods/functions?
> extract some arbitrary subset of the code and surround it with tests
You mean, like modules/classes?
- runtime state you need to initialize if your code is written in a stateful manner
- other methods/functions in different modules/classes the code you want to test calls out to
- the fact that method/function and module/class separation is (and IMO always should be) primarily driven by needs of production architecture, not testing, means that it may not be perfect for testing needs
Add to that the failure of testing tools and methodologies (like "don't test private methods" meeting "separate out duplicated or complex code into private methods"), and I feel the problems are real.
----
Here's an idea that just popped into my head right now: how about "hash-based testing"? You take a code you want to test, like:
private Integer foo(String bar, Frobnicator quux) {
String frob = quux.invokeMagic(bar);
return memberApi.transform(frob);
}
and turn it into: Method cut = <<
private Integer foo(String bar, Frobnicator quux) {
String frob = MOCK(quux.invokeMagic(bar)) AS("frob");
return MOCK(memberApi.transform(frob)) AS (frob.length() > 3);
}>>;
//continue testing cut()
The idea being, the compiler or whatever external tooling ensures that the original method in production code, and the inline-modified method in tests, are the same with respect to some equality/hashing function that always treats expressions "foo" and "MOCK(foo) AS(bar)" as equal.This way, you end up being able to mock everything precisely, inline, in whatever way you see fit, with your tooling ensuring that the actual code stays in sync with the test, since whenever the original method changes in any meaningful way, you'll fail the code "equality" test.
Might be a stupid idea, I welcome comments.
(INB4 testing to implementation instead of the interface - if you have to mock anything, you're already testing to implementation, and this way you can inject testing alterations precisely, instead of having to turn your architecture inside-out to support IoC/DI/whatever the current testing-enabling OOP fad is.)
Text substitution is impossible to reason about; magic test frameworks sound like a good idea because every kind of magic sounds like a good idea in isolation, but they'll always end up getting you in trouble. Plain old code, plain old functions, parameters, and modules, always win in the end, because every bit of brainpower you can save from understanding magic is brainpower you can spend understanding the details of your actual test instead.
That said, I didn't mean text substitution, but AST transformations (or rather, AST comparisons, since my example is defined in terms of AST equality with the MOCK magic thingie making the particular part of AST be ignored for the sake of comparison).
So really the idea that a good design is also testable is somewhat of an approximation of reality, It has some elements of truth but at some point making your code more testable actually makes it worse, often you need to make other compromises. Then the interactions of pieces of code typically happens in much more brittle system/integration tests. YMMV, this is just my experience.
A lot of interesting pointers in the replies, thanks!
When I was at google, one of the hardest thing I did was look at the marginal intern review scores and tried to pick out the ones from the big pile whom we should look at again and who should not be looked at. There were all kinds of crazy ass stupid interview questions that in my opinion were not very useful to classify capability.
Or they couldn't recognize failure
It's almost like the "problem" isn't the languages themselves, it's the tooling around the languages. I want to be able to do a lot more than "compile", "run", with my code... I can imagine machine learning driven tooling being able to automate a lot of the mechanical aspects of code writing beyond the simple generate a closing bracket that an IDE can do...