Correct. If you have to do research and experimentation before you can make good estimates, then you need to allocate time to do that.

I use the term discovery phase to describe this. You give the discovery phase a certain amount of time to complete, it should be from a few days to 2 weeks. The deliverable at the end of discovery phase is a document. The document explains what experiments were run and what we should do next.

This still rubs me the wrong way. I guess I personally don't like micromanaged environments. I don't want to have to explain every little step and report back all the time to people whose only interest is the deadline. I prefer to be treated like an adult who can be trusted to do the right thing to get to the goal quickly without being supervised at every step.

But I know plenty of people who need constant monitoring and handholding.

I can see GP's point and I don't see it as micromanagement. It's more like intensive investigation. I was "promoted" into management recently and I've had to change my view from "tasks getting done" to "people working together to complete tasks." Getting people to work together fruitfully is pretty difficult and even more difficult when you have a deadline you're shooting for. I don't care as much for reports back to me on progress, but I definitely like to see communication between those doing the task.

This also goes well with the rule mentioned in the article: always have at least 2 people working on a project.

People talking to each other is great. I like to bounce off ideas but I don't like to report to a manager who cares only about time all the time.

It doesn't make things faster but instead takes a lot of mental energy that could be better spent on solving the problem.

This sort of works if you have only a couple discovery tasks. Once you have a few dependent tasks -- that is, C depends on outcome/method of B, which in turn depends on A -- the branching becomes crazy.

Frankly if I'm going to spend a couple days on one aspect, I'd rather just implement that aspect.

I prefer to identify the risky parts and do them first, and refine estimates along the way. TDD helps because it keeps components isolated, so you don't have to build "in order".

Waterfall development with adequate time for planning/discovery up front does for sure give better estimates -- but those estimates are very expensive to get and are usually not all that reliable anyway, especially for multi-month projects.

> If you are doing things that require a lot of research and experimentation

In other words, "anything useful".