On the missed deadlines issue, that reminds me of a blog we posted a while ago about why we stopped estimating ongoing development on these types of code bases[1].
I also like to use the metaphor of a car that's stuck in the mud (or snow). The wheels Are spinning really fast and the engine is working really hard, but the car isn't going anywhere. It's only when you start trying to dig it out that you start to see forward progress, and even then, the progress is slow at first.
[1]: http://corgibytes.com/blog/no-estimates/productivity/custome...
>> did cause some missed deadlines early on, and getting all the abstractions correct still means basic functionality often doesn't come together until the last minute.
Yes, yes, yes.
>> I think most managers would be horrified this approach
Yes, been there, done that too!
http://www.folklore.org/StoryView.py?story=Negative_2000_Lin...
"Bill Atkinson, the author of Quickdraw and the main user interface designer, who was by far the most important Lisa implementor, thought that lines of code was a silly measure of software productivity. He thought his goal was to write as small and fast a program as possible, and that the lines of code metric only encouraged writing sloppy, bloated, broken code."
Requirements always change and information is never perfect. You can't always get everything right from the start because you might not even know what you're trying to get right.
If you get malleability right from the start, you'd expect _more_ refactoring, since it's easier to do when you inevitably want to. If you're going to spend effort on trying to get things right from the start, that's where you should focus: Minimize things that are slow change, maximize things that make change less painful.
Concretely, this means spend less time code reviewing local variable names and curly brace style; spend more time reviewing public APIs, data format definitions, promised external semantics, etc. with an eye for versioning, deprecating, evolving, etc.
1. As a die-hard function programmer believe most everything out there is designed poorly---not because everybody incompetent but because anything working with poorly designed languages and libraries must itself necessary be less well designed. Bad abstractions breed other bad abstractions! To write near-ideal software usual an excise of ripping up the foundations---so a lot of refactoring.
2. We learn good design best by refactoring. When righting new code, functionality and elegance are competing goals, but when refactoring elegance can be tackled in isolation. Furthermore, refactoring is an opportunity to prove (to oneself and others) that given abstractions indeed make code better. Finally, it's hard to distill good design to principles beyond "don't repeat yourself" and "make bad things impossible", but refactoring offers an opportunity to try things and see what works.
I've found it next to impossible to get it through ( some ) people's thick heads that refactoring is not a sign that something was done wrong the first time around.
>> We learn good design best by refactoring.
Exactly. This is often when the true value of a design pattern becomes crystal clear in an applied ( rather than theoretical ) sense.
Most of my side work is contributing to existing projects, but here's the stats for 3 school projects: 20,896 lines deleted for 30,342 written. 2 of the projects had fixed requirements from the beginning though one was a more free-form independent study. Additionally all 3 had only one additional author and I did a majority of the line additions and deletions.
I'm pretty sure these numbers are way untypical for this sort of work---with fix goals and deadlines with no more than a few months per project I assume most people delete pretty little, and basically only during debugging.
Certainly this refactor-happy approach did cause some missed deadlines early on, and getting all the abstractions correct still means basic functionality often doesn't come together until the last minute. But I still that the immense technical wealth does pay off as I get more productive the longer I maintain the project---the abstractions anticipate the sort of extensions I may later wish to add by laying a firm foundation. And the fact that the functionality does appear last second speaks testifies to the increasing productivity---longer hours or last minute panic doesn't explain it.
I think most managers would be horrified this approach, but if you're a real company planning on being around indefinitely and not some startup just hanging together or student, this method should be all the more fitting.