Your first mistake was misunderstanding the relationship with management.

Time estimations are bargaining, you ask for something and then get a counter offer from management, they don't care that the counter offer from them is delusional. The bargaining is the point, the less time you bargain for the more stressful the development can become. Don't estimate what it should take, estimate what it could take and bargain from there.

Get out of the company if it has this kind of management.

I hate to say it but from my 20+ years in the industry, this is the norm.

You are absolutely correct, but there are groups within companies that don't work like that. Finding them is the hard part :|

I think it mostly correlates with how technical PMs are.

People that were fairly technical prior to deciding to move to PM roles, those you can actually explain the context and reason with.

PMs that are more biased towards Business/Marketing are big on C Level networking etc., those are the ones that feel they are doing a good job if they try to bleed a stone with deadlines. They are adversarial like GP mentions and not someone you can reason with.

It is more common than not, but it is still best to avoid such places.

I'm a manager and when I ask my engineers for an estimate, I always, always increase it for my own purposes.

Indeed, management is very much about shielding the team, and getting rid of roadblocks.

These are the kind of managers I will go the extra mile for. Got my back I got your back and make sure you shine.

The other part is that management/PMs usually care about deadlines, not work effort (duration in man-hours, story points, or whatever).

Allow me to indulge in a fictional narrative:

You are handed "high level" (a.k.a rooted in the fuzzy non-technical alternate reality that is product management) requirements resulting in known unknowns and unknown unknowns. You're junior and naive and you put real tech-borne effort numbers in front of each task; or you're experienced and you put O(t) numbers on the first ones and random but high numbers on the second ones.

Anyway, the estimates end up landing past the desired deadline, management mutters TLAs such as "OKR" and whatnot, you'll have to justify why it doesn't fit, so you communicate work effort.

You are summoned into a retroplanning meeting to cull out stuff based on work effort so that it fits management's expected deadline, and try to pretend you're doing agile when it's all waterfall Gantt in disguise.

Project starts. Implicitly man-hours become wall-clock-hours, irrespective of other tasks such as CI/tooling/dependency breakage, code hygiene, or customer support escalations; your effort estimates became t{project start}+effort == deadline. You are now held accountable for the latter based on you communicating the former.

Time is now the scarcest of resources, you do your best, conjure magic, work extra hours, take shortcuts, and make it to the deadline, sacrificing body and soul. In a cloudy spreadsheet checkboxes have been checked, management is happy. Deployment is smooth but soon corner case bugs and escalations start trickling in. Management is either oblivious or not so happy after all.

Time for a new quarter to start. You know issues borne out of past quarter will rear their ugly head, and there's much tech debt to repay before interests quickly accrue towards tech bankruptcy. You're junior and swear to yourself to not make the same mistake, and you start inflating numbers; you're experienced and you realise the whole number dance is a sham as they don't matter anyway, but you still are required to do the dance because that's how the game is played, so you turn Scotty and just throw outrageously big numbers up in the air and just make sure the deadline is not too ludicrous, still hanging to the hope hoping that someone, somewhere, has seen the light of reason and built a company around it, and maybe you could go to these greener pastures; you're very experienced and you realise the only winning move is not to play, so you quit (not the job, the _industry_) and move to a remote plot of land with literal green pastures and a small cabin mountainside to raise sheep, or cut wood, or whatever, where the smallest time unit is the measurement error of eyeballing the zenith-relative angular position of a giant fireball 8.3 light-minutes away.

The correct response to that sort of bargaining attempt is "What are we dropping from the project scope?"

The tell is that as soon as they don’t like how things are going they flip from calling them estimates to commitments.

Also if someone’s estimates aren’t perfect, that’s something you can use to justify withholding salary. The game is fixed. It’s just a good deal less fixed for us than for the average person.

Withholding salary? How is that legal?

Well your responsibilities warrant a 20% raise but since your estimates are bad and your <bullshit other reason> was not 5 out of five, we’re giving you a 6% raise. Better luck next year right?

I don't disagree with you but I think people also need to understand that management often pick the team that provides lower estimates because team members will have to work harder to deliver it.

Management might not be so dumb. If someone said 2 months and you say 6 months, they might go with the person who said 2 months but will accept 5 months secretly.

I think at least a lot of them are rudely surprised.

If it’s a conspiracy not everyone is in on it, at least. There’s more an element of believing your own PR in many cases.

In my experience the 2 month version is so bad it’s still being hotfixed in production in month 8.

That's a success story.

The 2 month version could be sold at all, the 6 month one can't, so it's not worth considering its technical merits.

Then the client/stakeholder buys in, the sunk cost, well, sinks in, and things get fixed. Success!

The 2 month version experiences scaling issues at a 20% rollout, or takes 20s for a database query before returning a page.

In 6 months you need a better scaling version while the current version is creating cascading database failures; it's got terabytes of denormalized data that is out of sync and a 2nd team can't migrate the data in 12 months.

In 12 months, you find 5 other teams copied the 2 month solution.

In 2 years, you have 0 devs left that can solve the problem and a 1000% grown dev team that is great at the 2 month solution.

You ask the teams why they don't use the scalable solution and they say, "do we have any examples of that, the unscalable way is our convention"

>>>. The real problem is not with estimation but with the management denial.

^^ This is the truth. All estimation processes in any company are just ceremony. Management already has a deadline fixed. They will make you agree to their deadlines, even though whatever process stipulated by them says otherwise.

Not EVERY company. Just the ones that don't understand how estimation works... and unfortunately in software, that's still a very very common case. In more established kinds of engineering it works differently... which gives me hope that eventually our industry will understand too.

> Management already has a deadline fixed

The real problem is saying "yes we can do it all in that timeline" instead of saying "That's fine, but we won't get to everything. We will work with you to prioritise the most important things to get done in that time."

I know it's painful, but just hold that line. It's not lying or negotiating or padding; in fact it may be the only true statement you can say.

If a civil engineer were building a bridge, and couldn't do it safely in the budget and timeline, they'd say so, and the project manager would nod and diligently write that down. We have a more subservient relationship with project managers in software because we choose to.

You have to get them to take that perspective. What happens is "we want all this stuff how long is it going to take" and then they fight that number down while not answering any clarifying questions. So you have to sumo wrestle a bunch of assholes twice your weight into even framing the context correctly before the conversation you recommend can even take place.

> They will make you agree to their deadlines, even though whatever process stipulated by them says otherwise.

I'm not sure how. I mean, if I know it takes X time to do something, what would make me to commit to Y<<X time when I know it will fail and will be made my fault? If they want to record it as a deadline, so be it, but if it's an impossible deadline I'd make every effort to be on record that I said it from the start that it's impossible.

Management denial is not universal. It can't be - companies that have realistic management should out-compete companies where management is politics and unreality.

The problem is that management unreality can take a decade or several to destroy a place. But when you see that kind of management, know that that's the road they're on. That's your cue to start looking for somewhere with management that at least tries to live in reality.

I hope you are right. I have never been in or seen a workplace where management is not playing the 'get the order at (nearly) all costs - let others handle the problems' game. Again, I hope I am wrong. But these type of companies seem to outcompete all others. The only unicorn in a decade in my geographical area, is a company that plays this game very very well in the IT sector. As far as I have seen, they play a variation called 'the customer takes the problems (and the full financial impact). Due to building a good set of core competencies, that the competition mismanaged over years, they can get away with this strategy.

You’re forgetting that in companies with the management properties you say are bad, those properties may cause actual value to be delivered faster, even if it’s at long term cost. That value can accrue, gain interest, and that company can be more successful than the one with the management properties you say are bad.

I think you’re conflating success of a company with the pleasure of its employees. These 2 factors are usually (not always) opposing not correlated.

This also makes fruitful grounds for scapegoating. 1.5 years into the "6 month" project, they can reorganize because of the catastrophic failure.

Then the new organization can say it wasn't because they misestimated, it was just the poor organizational structure before.

Obviously this new structure would have achieved the result in the time allotted.