In other words, when it turns out the estimate was too optimistic one can either:
- allocate more resources to the problem,
- make compromises about quality of work,
- miss deadline or
- adjust scope.
The first solution is rarely possible (or doesn't help because new hires take time to be productive), the second has unwanted side effects, so the last two are the best options... Pick your poison I guess.
- (2) Compromising does not always compromise quality of work. You can always cut scope as you mention in (4) and ship less features at a higher level of quality. Although what I was getting at was more that we have a tendency when given time to go about refactoring and rearchitecting things that don't necessarily need to be done, or introducing unnecessary / premature abstractions and optimizations. Deadlines tend to curb instinct to do that.
- (3) You can miss deadlines, but that's not a clean win either. It hurts morale, allows for feature-creep (oh you're not shipping next week, well boy do I have some extra things you could do with your newfound time) and hurts relationships with teams depending on what you're setting out to deliver all across the company.
- (4) Adjusting scope can make sense, so long as you're very good at figuring out what doesn't need to go out. Not every team is.
tl;dr: Shipping the right 70% of the feature set at 100% quality without premature / unnecessary optimizations and abstractions can be pushed along by aggressive timelines. This must always be balanced with sustainability.
It looks like a few people have reached somewhat similar conclusions and create frameworks around this idea (Agile, Extreme, TDD etc.) to formalize the processes. But it perhaps makes sense to realize that they are just that: processes and heuristics trying to make a hard problem (delivering software predictably on schedule) more manageable.
That said, I've always been able to provide fairly good estimates for how long work will take, something that I got a reputation for doing well.
In my experience, that's not really true; I find value in time estimation to come from actually scoping the work again and probing for traps or pieces of complexity that weren't obvious in the first estimation passes. It's not true that software takes as long as it takes. Software projects expand to fill the amount of time allotted. Holding yourself to deadlines creates room for compromises and creativity.