This approach of raising the bar for introducing functions might do well with my "trace tests". I'm going to try it.
[1] Sorry, I've temporarily turned off my site while we wait for clarity on shellsock.
You're getting more coupling to a codebase-wide object then, which goes against some principles, but it allows testing by doing things like
function awesomeStuff(almostAwesome) {
MoreAwesomeT f1(somethingAlmostAwesome) {
TestSingleton.emit(somethingAlmostAwesome);
var thing = makeMoreAwesome(somethingAlmostAwesome)
// makeMoreAwesome is actually 13 lines of code,
// not a single function
TestSingleton.emit(thing);
return thing;
};
AwesomeResult f2(almostAwesomeThing) {
TestSingleton.emit(almostAwesomeThing);
var at = makeAwesome(awesomeThing);
// this is another 8 lines of code.
// It takes 21 lines of code to make somethin
// almostAwesome into something Awesome,
//and another 4 lines to test it.
// then some tests in a testing framework
// to verify that the emissions are what we expect.
TestSingleton.emit(at);
return at;
}
return f2(f1(almostAwesome));
}in production, you could drop testsingleton. In dev, have it test everything as a unit test. In QA, have it log everything. Everything outside of TestSingleton could be mocked and stubbed in the same way, providing control over the boundaries of the unit in the same way we're using now.
I've had to change an implementation that was tested with the moral equivalent to log statements, and it was pretty miserable. The tests were strongly tied to implementation details. When I preserved the real semantics of the function as far as the outside system cared, the tests broke and it was hard to understand why. Obviously when you break a test you really need to be sure that the test was kind of wrong and this was pretty burdensome.
"..trace tests should verify domain-specific knowledge rather than implementation details.."
--
More generally, I would argue that there's always a tension in designing tests, you have to make them brittle to something. When we write lots of unit tests they're brittle to the precise function boundaries we happen to decompose the program into. As a result we tend to not move the boundaries around too much once our programs are written, rationalizing that they're not implementation details. My goal was explicitly to make it easy to reorganize the code, because in my experience no large codebase has ever gotten the boundaries right on the first try.
I think maybe the answer is that you want to do the development all piecemeal, so you can test each individual bit in isolation, and /then/ inline everything...
That sound like it might be effective?
Current experience indicates however that such end-product testing gives you no real advantage to finding out where the problem is occurring, since yeah, you can only test the whole thing at once.
But the sort-of shape in my head is that the god-function is only hard to test (after development) if it is insufficiently functional; aka, if there's too much state manipulation inside of it.
Edit: Ah, hmm, I think my statements are still useful, but yeah, they really don't help with the problem of TDD / subsequent development.
I’m not so sure. I’ve worked on some projects with that kind of test strategy and been impressed by how well it can work in practice.
This is partly because introducing a bug rarely breaks only one test. Usually, it breaks a set of related tests all at once, and often you can quickly identify the common factor.
The results don’t conveniently point you to the exact function that is broken, which is a disadvantage over lower level unit tests. However, I found that was not as significant a problem in reality as it might appear to be, for two reasons.
Firstly, the next thing you’re going to do is probably to use source control to check what changed recently in the area you’ve identified. Surprisingly often that immediately reveals the exact code that introduced a regression.
Secondly, but not unrelated in practice, high level functional testing doesn’t require you to adapt your coding style to accommodate testing as much as low level unit testing does. When your code is organised around doing its job and you aren’t forced to keep everything very loosely coupled just to support testing, it can be easier to walk through it (possibly in a debugger running a test that you know fails) to explore the problem.
Local function bindings declared inline perhaps? It seems to me you could test at that border.
But, it sounds like what you are dealing with is not inline blocks of separable functionality. Sounds like a bunch of good-old, giant, messy functions.
I'm not fanatical about TDD, but in my experience the trajectory of a design changes hugely based on whether or not it had tests from the start.
(I loved your comment above. Just adding some food for my own thought.)
I'm still not sold on the benefits of fine grained unit tests as compared to having more, and better, functional tests.
If the OPs 1k+ methods had a few hundred functional tests then it should be a fairly simple matter to re-factor.
In "the old days" when I wrote code from a functional spec the spec had a list of functional tests. It was usually pretty straightforward to take that list and automate it.
Just so long as the outer function is testable and side-effect-free.
I don't think that's the kind of "inlining" being discussed -- to me that's the sign of a program that was transferred from BASIC or COBOL into a more modern language, but without any refactoring or even a grasp if its operation.
I think the similarity between inlining for speed, and inlining to avoid thinking very hard, is more a qualititive than a quantitative distinction.
I think what's being discussed here is quite either of those - this seems to be "inlining for visibility" and possibly "inlining for simplicity".
My first best-seller was Apple Writer (1979) (http://en.wikipedia.org/wiki/Apple_Writer), written in assembly language. Even then I tried to create structure and namespaces where none existed, with modest success.
Now I have to spend days and possibly weeks refactoring dozens of functions and breaking them apart in to managable services so we can not only use them, but also extend and test them.
I'm afraid what Carmack was talking about was meant to be taken with a grain of salt and not applied as a "General Rule" but people will anyway after reading it.