>Labour estimates are crucial for business, even if they're bad - the goal is to improve iteratively over time and gradient-descend towards an accurate picture of resource allocation.

I have two problems with this.

One, almost all work that is easily estimated falls into the realm of trivial work. Most work is not trivial given the rapidly changing work environments. If it isn't the tech itself, it's the people around you and above you. If it isn't the people, it's some context which in common human fashion, has become overcomplicated to high heaven.

Two, because things change so rapidly and riskmanagement is the way it is, we are in a perpetual state of businesses trying to pressure one another into getting the best short term deals at the cost of almost everything else. How many times has a deadline not been met with it being critical? Not that often, most things can wait another month. So why do we keep making unrealistic deadlines? (Semi-rhetorical)

That's so true, and why should one try to estimate non-trival work that we haven't done before, and make those estimates deadlines? Or worse, a higher management sets a deadline because of something (for example: launch advertising has already been paid for, so you must get it done by October or else) and you're now tied to that. I think it is premature optimisation to estimate absolutely every fine-grained task and I feel it's born out of the desire to look effecient.

It makes sense to estimate R&D, it also makes sense to have limits. (Eg. some startup wants to solve global fidgeting, they raise a lot of money, and they have 3-4 ideas, so they allocate half of their money and one year on their best idea, then half of the remaining and half of a year on the second best, and ...)

If there's some set date (eg. demo/launch at big industry expo) it makes sense keep that constraint in mind, plan work in small chunks, iterate, and always try to deliver tangible things needed for the demo ... and it also makes sense to periodically check in (let's say at the end/start of every iteration) how the demo is coming along ... and it again makes total sense to adjust course if it seems that progress is better/worse than expected ... and it makes sense to try to do this tracking and estimation in some consistent/rigorous way, instead of just gut feeling.

"And that's how story points were born". (Which is just a fancy name for reference class based forecasting - https://en.wikipedia.org/wiki/Reference_class_forecasting )

And, again, if there's enough reliable data it's possible to give educated guesses about how the remaining tasks compare to the already finished ones.

...

Now, doing this top-down, from day 1, expecting 100% accuracy, and whatnot is complete business madness, and anyone who does that needs to be thrown out of a fucking (ground floor) window again and again until this gets through their (apparently) very thick skull.

...

I've seen this done well. (Not the defenestration.) And seen this done horribly. The "methodology" is not magic, and it definitely doesn't make sense if the people trying to "do" it are senseless imbeciles forcing things they don't even understand on people whom they also don't understand.

But most part of work that got funding is trivial. Frontend, backend, CRM, backoffice, data this, data that, ETL, audit, backup, devops, cache, mobile, blablabla.

Usually what's not trivial is how much to build and when (which is product design).

> it's some context which in common human fashion, has become overcomplicated to high heaven.

yes indeed :)

If business (and/or management) is too neurotic, that definitely will fuck things up and it doesn't matter what methodology you try to manage projects with.

I used to scoff at the first dictum of the agile manifesto (individuals and interactions over processes and tools), but it fits exactly these problems. Business has to plan and build on what Engineering already does, not try to force things down onto it. Management has to work with the people, not try to force the people into their own crazy world.

And if they don't, then out of self-preservation, leave.

I think you're missing the part where people who don't do software are asking people who do software.

I've literally had clients go "Hey, can you add search to the platform? Can't be that hard, it's just a single input field on google dot com".

Your task is to tell them that this is too big to estimate and needs to be specified better and broken down into managable chunks. Well ok, in this case, just telling them to sod off would've been smarter, but that's a different story.

I'm happy to provide whatever estimates are desired by the business, and I'm sure all other programmers would too. But...most of the time I'm estimating something that I haven't done before (or for a long while), so they're bogus!

The common pattern I've seen is that everything is about estimates, which end up becoming deadlines. The management worry about them to the point of bloody-mindedness, at the neglect of gathering actual requirements from the customer. In my company, I work in a software department (I suppose we're an internal agency with just one customer). But, we can't speak directly to the customer and instead speak to another Project Manager (apparently because of "SaFE Enterprise Agile" policies), so we can't ask the customer what they truly need, and so the product isn't accurate, and neither are those estimates!

Estimates become deadlines because most managers don't understand the terms 'statistical variance' and 'sample size', not because the estimates are made. ;)

Entering a contract with a client based on an estimate without a large margin for error is a statistical failing, not a strategy one.

That said, I still think waterfall and other related variants are overwhelmingly the way to go. I've done both styles and the difference is night and day.