Seriously though. It's always a time tradeoff. How much time do you want an expensive developer investigating how long this problem will take. Will they learn something they can use during development.
I always follow up my above quote with the unknowns and how hard I think they really are; what are the possible risks; _and_ I include the experience of the developers working on the problem; our ability to bring in other devs who understand this better etc etc. I like to make sure whoever is quoting this problem out understand everything as well as we can in the given amount of time.
If that number is too high, then that’s a sign to management that the work needs to be changed. The only way to determine that is to estimate it then make that call.
And only works if management wants an estimate. More commonly, management wants a lie they can tell the client.
That’s absolutely preposterous. As a senior engineer, I have seen enough that I have at least a high-level idea of what most of the dev tasks and stories I’m estimating are. I don’t always have the specific details that may change the estimate, but I also have a whole team of other people who have seen even more things and can inform that opinion.
The point of refinement sessions is to share this information with everyone else so an estimate can be created and if needed, tasks can be better defined, split up or merged.
> And only works if management wants an estimate. More commonly, management wants a lie they can tell the client.
Wat? Estimates is used for plenty of reasons that have nothing to do with external clients.
How do you think your manager, their manager, etc knows what is going on and approximately how long it’s going to take? They’re going to be far more disconnected from the work than the people doing it day to day, so they would generally have no clue how long something is estimated to take without someone telling them.
You are misunderstanding your job. You are a subject expert and paid accordingly. Answer with what the unknowns are, an optimistic and pessimistic take on them, what you could do and how long it would take to lift them.
If you don’t, your boss is going to ask you for exactly that in a roundabout way and micromanage you because you are forcing them to do your job.
It still baffle me than approximately half of developers don’t understand things which are obvious to any seasoned blue collar worker.
In software, on the other hand, we often have most training being taught by professional academics who do not understand and are not qualified to teach the parts of the job that don't have a straightforward academic analogue. Combine that with a business environment where things like budgets and cost control are often only theoretical, and all but the most gross liability for defects due to poor workmanship has been waved away by EULAs, let it marinate for several decades, and... yeah. Not only do people not know these things, but I'm not entirely clear on how or why they should be expected to learn them, either.
If he really needs an estimate he'll tell me to start digging into it right away, and I'll let him know after a bit of time how it's going. If there are still large uncertainties and my boss still wants this completed by some specific date, we might try to explore alternative solutions.
In the above case, I didn't get that time I would need, they wanted the estimate to include that time I would need to dig into it. But that's above my estimation powers. I didn't even know if what they asked was possible at all.
They went and asked someone else who did guess some number of weeks, but the work never got priority so we still don't know.
Yeah that ain't gonna fly with me.
Did you say that
1) Without knowing further best case is it takes N time, worst case M time, and if there are certain obstacles it could be completely unfeasible. 2) To get more accurate time and to understand whether it's feasible I would need O time to figure that out first, and I can give you the more accurate estimate then.
The problem is when we assume everything is a known. I try not to do that and I try to set that expectation with my manager. If everything had an easy answer we wouldn't need highly paid problem solvers.
Well, that's the problem right there. An estimate is not a promise. It should not be treated as such.
It's not an apples to oranges comparison. You give me an architect's blueprint and guys who build to the legal building code, and I could probably get a house built. Expensive and painfully, but it can be done.
I don't even know what the coding analog to an architect's blueprint is. They are fundamentally different jobs.
Yes, I work as a delivery lead nowadays. Producing and respecting estimates for software projects is literally my job.
I routinely fire developers who pretend they can’t do estimate and strangely the ones I keep are actually not only able to do it but good at it.
I've met your sort before. You're a net waste of resources in an org: instead of developers getting things done, they're dealing with you asking them to do your job for you, i.e. produce estimates for the things they'll never actually get around to doing (because of producing estimates).
Your username would be accurate if s/My/Everyone's/
If it helps you sleep at night, sure, assume I’m a net waste of ressource. Good luck fighting with the other seven teams for some time with the shared senior UX expert.
If you rarely eat your time buffers, great. If you do that often, maybe you just coinflip-fix deadlines this way and maneuver between them under the guise of successful estimation.
You always get what you ask for. Like sure I can estimate. If you ask for a web server you'll just have that. It won't be secure. It won't be fast. It may work. It will certainly pass the tests you think of.
Is that actually good? No. The real estimates get rejected. Everyone cuts corners. When something does happen e.g. a hack occurs the engineers not these managers get fired :(
My optimistic/pessimistic takes at my last job would look something like "roughly a week to roughly 2 years"
Depends on how deep the yak shaving goes, and you wouldn't know until you got halfway through fixing the problem.
Often I could take an educated guess as to which side of the scale it came down on, but I've definitely started pulling on a string inside of the codebase and it just kept going and going and going. All of those baked-in assumptions that other developers unconsciously made over the course of a decade can pile up into a mess that you just can't implement the feature/bugfix without doing a pile of potentially breaking changes that need to be shipped in major versions.
Generally if I worked the problem for a week then I could give you a much better estimate of when it would be done, and often that estimate was "it's done now" after a week. But if I hadn't looked at that particular subsystem for a few years, I couldn't trust my memory of it up front and had to do exploratory surgery first.
(Really exploratory surgery is a much better metaphor for what you're going to be doing, and surgeons can routinely find surprises -- "oh look that barely perceptible shadow on the x-ray turns out to be a stage 4 tumor...")
Yes, the codebase was horrible, yes it needed to be cleaned up, no we didn't have the manpower to do that and it would have taken me 10 years to rewrite it all.
Quitting of course was an excellent solution.
You're lucky if your management chain allows you to work on problems for as long as a week without giving them an estimate first.
Add a small feature to an existing system. This estimate can be anywhere from several months to a day. Most of the error bars are org and system specific rather than anything to do with the feature. The exact same feature built by the same person in different orgs and different systems can be opposite ends of the range.
Seldom do people have the contextual understanding to provide an accurate narrow estimate, particularly since both org and system are in flux. They change constantly. Unseen landmines potentially await in both.
> If you don’t, your boss is going to ask you for exactly that in a roundabout way and micromanage you because you are forcing them to do your job.
And such orgs take forever to get anything done.
> It still baffle me than approximately half of developers don’t understand things which are obvious to any seasoned blue collar worker.
Software is probably the hardest kind of engineering you can do. All the system constraints are virtual. Just consider comparing a dirt ground for consistency and behavior vs libc... 304 stainless vs react.
You can go deep in both, but a shallow understanding of one takes you much much further than a shallow understanding of the other.
That’s extremely untrue in my experience. Sure, hard to interface with systems are a thing and organisational roadblocks happen but an onboarded senior team member can tell you what’s going to take days and what’s going to take months or they have no business being senior in the first place.
I feel like my answers in this discussion have been particularly harsh but I would really like people calling themselves software engineers to start acting like some. I think my view on this subject has been strongly shaped by spending most of my career doing software development for industrial companies: people who claim software is particularly complicated just don’t understand and respect how complex the rest of an industrial project is.
You're making an unwarranted assumption here. There's many pieces of software that have no development team at all. There's no one to ask; the last person to touch this software left the company three years ago. All they can tell you about it is how they use it now.
It's likely there's at least someone that can explain their business process to you, but identifying them and receiving enough of their time to piece the picture together is another story. And maybe, if you're lucky, that person can tell you what the business process was at the time the software was written, and what has changed since then.
But this process can take weeks, and most of the time you're expected to provide an estimate before even talking to anyone from operations.
Sure, for internal software with an active maintenance/development team, informed estimates can be given with a relatively high level of confidence. But as an outside consultant working with non-software companies, all estimates are completely made-up based on your reading of the buyer's budget: there are no other data points to base your estimate on.
Development by outside ressource happening on custom pieces of software from non-software related companies is such a niche it might as well not exist. I don’t think it’s a very useful datapoint in a discussion about estimation in software development. Plus, as someone contracted, I’m fairly sure you actually have a rough idea of the effort involved or you would have become insolvent already.
And yes they are still solvent.
As others mention there are lots of unknowns in an organization.
I've been in places where the other team involved is across the globe in a different timezone and isn't even actively willing to respond and yet we continue...
There are also cases where lots of parts are outsourced and you're lucky to get a reasonable response i.e. changing part x requires vendor a, b, c & d, which have an SLA 1-8 weeks and so you're already starting 4-32 weeks without even working anything else out. If you're really lucky it could still all be done in 1 day.
Not sure if you're harsh as you say or just lucky. Not everyone and every team is well organized.
Yeah, most of the time my boss just needs to know if this job is a few hours, a few days or a few weeks. I never answer with exact numbers, but a range. If there are some major unknowns I let him know. If I made some important assumptions then I let him know.
Sometimes I'm wrong. Most of the time my estimates are sufficiently accurate.
Sometimes the customer really needs this done before some deadline. Then it often turns into "how can we make this happen before then". After a bit of back and forth with some added assumptions and cutting functionality, we usually end up with a solution that can be realized in time.
Assumptions can range from “if this API can be consumed in a usual manner and is well documented” to “if these clarifications are made in time” to “if the domain experts test the feature on our dev environment and give timely feedback”.
I try to communicate clearly that it’s almost always a two way street.
Yes I’m the technical expert, but they are the domain expert. I need their understanding and feedback to make something that’s useful.
The best clients understand this very well. Those also get priority and sharper estimates.
As a manager now, I get scoffed at if I even hint at a deadline. Excuse me what? We have a limited budget, a limited runway, and we need to prioritize which product bets we are going to make. This seems entirely logical, but a lot of devs I work with now find it impossible. They just enjoy tinkering away with whatever thing interests them, and delivering value only means what they personally find valuable.
Personally, I find it a bit pathetic and childish. I say this as a long time software engineer who knows perfectly well how this business works.
That said, however, The problem I run into during practice is dependencies— people want estimates to integrate into systems you have never heard of or used before. Not only that, there is no in place architecture or design before estimation. Imagine a construction company estimating a house without knowing how big it is or without having blueprints.
Often times software development is designing and architecting the system — not just implementing it. And that is the expensive part.
Say that the estimation can depend on the complexity of the system. I would think worst case is when the system has these A, B problems, then it would take N time. Otherwise best case it could take M time. If we want more accurate idea, we can spend O time on it to figure out what it could be.
How does this business work?
Why is software special? This happens the same in construction and other industries. The estimates usually make 0 sense and there are often huge delays. Same as store openings, film production, etc.
There's a problem with tinkering and devs going off in the wrong direction but on the flip side when people say that something is done in an hour or a day often never happens. Managers and agile practices that want tasks broken done to the smallest atom just messes things up. To make things work product often never has accurate requirements to that detail either so that complicates things even more. The requirements change on the fly.
In a healthy culture it should be possible to have a back and forth about how accurate an estimate is likely to be, but when engineers insist that all estimation is useless you can imagine why other stakeholders may lost trust.
As a developer I never get “that’s a useless estimate we can’t prioritize based on that/tell the customer that”. It’s a perfectly good estimate. It tells everyone there is a lot of uncertainty. The low lower bound indicates there is a chance it might turn out to be cheap (so maybe worth taking a gamble and trying it and giving up). The big variance indicates it could/should be given some time to prep/investigate in order to narrow the estimate.
When there are unknowns, make a wide estimate. Convey the uncertainty. If the unknowns are ridiculous, make a ridiculous estimate. Even “5-200 days” is useful because something else with less uncertainty can be chosen instead.
For context, my team is in exactly this situation with several silicon bugs we found last month.
I'm running through a similar situation now, where I'm somewhat adjacent to a client and a dev team. The client has a list of tasks he wants done, and he'd like banded estimates for the effort the dev team thinks it will take to complete each.
The developers are obliging, but one of them complained to me privately "why doesn't he just give us 2 months of budget and we'll get through as much as we can?" - which is an interesting statement. From the developer's perspective, this is ideal. They are very good and work hard and have integrity, so I'm sure something like that would work, from their perspective. They can just 'go' and get stuff done.
From the client's perspective though, it just raises a bunch of uncertainty. He wants to know what he could get out of his 2 months of development budget. If one task took all 2 months, and the others (15 in this case) didn't get done, is that OK? What if some important task wasn't that much effort? You'd potentially want to priortise that over a bigger item.
Just saying 'give us two months of budget' doesn't take the other party into account, and what they're trying to balance or plan for. I've sat on both sides of the table, and honestly just asking for a budget with no actual commitment to completion feels selfish in my opinion. Discounting their opinion because they're 'business' people and not developers, or treating them like an enemy to fight against won't win you any friends or long term relationships.
The irony is that at least some of the time, a "safe" estimate will actually make the project take longer. Work naturally expands to fill a deadline, and not having a deadline but instead frequent progress checkins with an expectation of foward progress can result in more getting done, because it changes the conversation from "how at risk is the deadline" to "how much happened in the last week and what are the next steps", forcing progress to happen continuously rather than one big bang at the end.
I also often hesitate to give estimates / establish deadlines because it tends to create an unhealthy confrontational atmoshepere, where developers are incentivized to not incorporate feedback or make any changes even if they are the right thing to do, because it would put the deadline at risk (but if they incorprate allowances for these things during estimation it's considered sandbagging).
That’s probably the most important thing I’ve taken away from software project management techniques - everything they do is hurting them, not me. If somebody insists on shooting themselves in the foot, and they’ll shoot you if you try to stop them, you just sit back and let them shoot themselves in the foot and shrug your shoulders when it happens.
At the end of the day, these estimates are helpful but maybe not in ways that are intuitive from a developer’s perspective. They’re useful in weighing priorities, judging the uncertainty, and identifying where to fill the gaps in understanding, time-boxing research and scope (e.g. do we really need to spend 6 weeks reading release notes?), and help communicate to outside stakeholders. We see the successes of this all the time with big game releases, product launches, movie releases, etc. It’s a high level measurement of progression and devs need to be a part of that planning process. It gets easier for senior engineers to do this efficiently with experience. It’s not necessarily an exercise that a more junior engineer will a) get right consistently or b) think is valuable at all.
1. Coordination of external dependencies. Stuff like product marketing, sales training, documentation, etc. as well as sometimes things external to the company like customer commitments or sales commitments. These are for the most part legitimate, although in many cases the deadline doesn't need to be known at the outset of development, and too many of these is a very unhealthy sign.
2. Accountability / improved productivity - This usually manifests as an exec feeling that engineering is running too slowly / inefficiently, and by asking for deadlines (and often compressing them to "push" the team) they can improve productivity. These are almost always in my mind completely counterproductive, and there's much better ways to accomplish this.
It may feel good to have some of the times quantified. But... If your estimates are literally fiction. It won't stop being fiction in front of the client. You should really spend that energy that went into the estimate for something useful instead.
Not to say you can't derive some useful information from a short planning session. Just don't expect anything other than thumb to the wind guesses.
Maybe the solution here is "stick to your guns", but that can have negative career implications.
You can’t pull a different estimate out of thin air. And zero people will expect it.
The developer needs to say “I can say 10-100 days right now in this meeting, but given just 1 day of investigating, I’ll narrow the estimate range by an order of magnitude”.
As a PM, I couldn't possibly agree more with this. When you give me 5-200 days instead of "we can't estimate this", you've started the beginning of a productive conversation. I can ask you questions about why the range is so big, and you can explain to me what steps we can take (research, narrowing scope, etc.), to reduce the range.
Ordinary LoB staff apparently are superstars that have their schedules locked in months into the future, but consultant SREs costing thousands a day can have their time wasted on a daily basis trying to determine how long it will take to complete a task that hasn’t even started yet.
This has literally been my last two months. The dev team lead is asking me — an outsider — how long it’ll take his team to finish something. Not because I know better, but because he doesn’t know either and prefers to make it my fault because I’m expendable.
1. Talk to the person who'll get angry at you when the UAT testers are brought in at the wrong time.
2. Make it clear to them, with a paper trail, that this is an impossible task and you can either book in a best-estimate with the likely outcome that the UAT testers will arrive and there will be nothing to test OR that you can book conservatively and there is a risk the project will be delayed waiting for UAT. They can pick which risk they want to bear.
3. Put in formal feedback, as loudly as possible, that booking in UAT for a specific time introduces obvious risks and someone needs to optimise the process to be more flexible otherwise it will delay projects & waste the businesses' money.
If you're the user or representing the user, shouldn't you be doing the UAT?
You can pull a different estimate of 20-40 out of your behind when pressured to narrow it, and make the deadline 40 days. That might be a 50/50 hit/miss the deadline, with a worse case of running over to the full 200 day worse case (or even worse - even the 200 was a guesstimate). In some cases that's fine too. The decisionmaker might just want a date with 50% chance of hitting because it's better to land a sale and apologize for the delay, than to not land the sale and go bankrupt.
The correct course of action though is to not set the deadline in that meeting. The 5-200 day estimate is a way of explaining "Either don't take this on at all, or at least don't try to schedule anything urgent based on it".
In that case as a developer I'd suggest "Let me look at this for one afternoon, and I can return with an estimate that's 1/10th as large. Then let's meet again ad prioritize some work, schedule UAT and so on". PM's often look really surprised when they realize that all it takes might be to check if there is a functionality available already or if it needs to be implemented - which represented the difference between 5days and 200. If I go off and find it's already there, the estimate is 5-10, if I go off and find there is nothing and it has to be built from scratch, it's 150-200. But without that knowledge: 5-200.
Or, if a 20-200 day estimate is actually due to real implementation uncertainty, I'd suggest "This is potentially large and uncertain. Let's try to break this apart if necessary into something that has a 10-20 day estimate and some parts that have a longer horizon. Schedule the delivery/UAT of the first functionality in 20 days, and we'll by then have a good estimate of when the remaining work will be done".
If someone in my organization would try to convince me to pull a stupid estimate out of thin air, then pressure me to haircut that estimate, then schedule something based on it (such as a delivery), then pressure me to work overtime to meet it, then one of us would be leaving the org very soon.
I quite literally quit my last job in February because of this shit[0]. Stop. No. These are not useful things for developers, much less senior developers, to spend their time on. 95% of my job ended up being coming up with estimates. My top deliverable to my management chain was estimates. They would only care about the actual development work insofar as it validated or invalidated the estimates deliverables.
Hard facts: If time is such an unimportant resource that you're willing to waste most of it doing "research" and worrying about scope, then you don't actually need an estimate. For reporting purposes, just BS one yourself, that's literally your job as a PM.
[0] It was well-paying, I was basically fucking around every week doing whatever I wanted, yet this issue alone outweighed all of that and was enough to make me quit.
I do that and I've become exceptionally good at blaming the business persons for the outcome, making them admit their guilt in public and all that in what seems a friendly, positive and constructive manner and using business motivation and explanation. They never understood what hit them. Every programmer can do that if he or she learns a bit of psychology.
If some manager made a promise to a client out of his/her ass without asking us, and if the PO pressures us towards meeting an unrealistic time-frame, when that explodes I just explain in a public meeting where upper management and all stake holders take part why it was wrong to promise that, using arguments and business considerents because bussineses people don't care about tech considerents. It usually results in that person admitting in public their mistake and even saying they are sorry.
I don't do that to humiliate people but to teach them to ask the team implementing the feature to estimate or to say if that feature is even reasonable or possible given current state and resources.
The estimates aggregate to a number more accurate than each individual. The manager cares more about the sum total than each individual one. Many can even slip without affecting the end date.
This isn't so hard, if management and developers didn't treat each other as enemies. Each can get wrapped around the axle concerning each deadline, rather than the real goal of allocating resources across the entire scope.
The difference is, the law says if a contractor signs a contract to do something at x price, they HAVE to do so or suffer consequences. So contractors go broke/bankrupt, lose their licenses, or figure out how to bid correctly enough they can either weasel out of it if it goes sideways or they have enough wiggle room to deal with the inevitable problems.
Software Engineering Consultants have the same situation. But directly employed software engineers are in a different position of course.
One thing that typically makes this worse, is the implicit assumption that planning/quoting/estimating is ‘free’ work, and since it CAN make things easier to deliver, therefore we should do more and more of it as, since it’s free, it will solve the problem no?
And their disciplines involve massive capital outlays which can’t be refactored in a cost effective way, or at all really.
Not defending the ‘no estimate ever’ approach, it doesn’t work. They are different though.
Tell me, how accurate were the estimates for the launch date of JWST?
I’ll wait here while you look up how many times and how many years that “slipped”.
I bet some dumbass NASA manager asked the team to schedule the launch date back in the 1990s.
It gets even worse when you are in a line of work where the estimates are part of some kind of sales pipeline. Then you aren't just estimating based on effort required, but also against a likely number that the customer will ever agree to.
However, I don't think this idea would be very popular because it removes the manager's ability to control the presentation and play politics.
Except instead of anonymity, the devs have to actually discuss until they reach consensus.
The correct solution here isn't just to insist estimation is impossible and be offended that someone would ask for it - that just misunderstands how businesses work.
The correct thing is for either of those parties to say, "Okay, I understand it's not estimable at the moment. What are the steps that we need to take in order to get a reasonable estimate?"
A good PM will do this, but where there is a bad PM, a developer can step in and say the same thing. In your scenario, "The developer says “there are too many unknowns to provide any sort of estimate”." What the developer could say is, "There are too many unknowns to provide any sort of estimate - let me give you a few examples. <examples>. What I can do is come back to you after <amount of time> with a list of the unknowns and an explanation of the work required to evaluate them, so that we have enough information to estimate. Once you have that list, then you can decide if we want to proceed with the work to understand those unknowns."
I totally understand why developers get frustrated when they're asked to give estimates for things that can't reasonably be estimated with the current information available. But the flip side is it's also very reasonable for PMs to get frustrated and ignore developers who insist that estimation is an impossibility, because that's virtually never true - maybe it's impossible to do now, but there is a path to make it possible. If you work proactively to expose that path to the PM, it'll generally result in everyone being happier.
Sometimes we don't know what we don't know. If we don't know what we don't know now and if we don't know what we'll find we don't know later, then we don't know how long we will need until we'll know.
Uff. Sorry. If I had more time, I would have written a better comment.
If it's the latter case, then a good developer can describe the immediate steps they'll take and what questions they expect to answer. Whatever the case, there is always some concrete next step you can take to which you can assign a reasonable estimate of time. That might just be that you'll spend one day perusing some documentation, and the only outcome you expect from that is that you'll develop a reasonable enough understanding of how large/complex the documentation is, such that you can come back with an estimate of how long it will take to find and carefully read all the relevant sections of the documentation.
That's totally fine - if it's the sort of wildly unknown problem that you're describing, then I recognize we'll have to take steps to understand the problem. There's still never a case where it's impossible to create an immediate plan of action and understand roughly how long that will take (unless you are simply unqualified to take on the problem).
I had a boss loved the phrase, "a date for a date." He just meant that if you can't give me a date for when the thing will be done, then you need to give me a date for when you can give me a date for when the thing will be done. Can't do that? Date for a date for a date. However many levels we have to go, you're not getting out of whatever meeting we're in until you can give me a date by which you'll be able to come back and give me another date.
Right now I’m replatforming a legacy app. It’s undocumented, developed by dozens of staff — all of whom are gone, and full of crazy bugs and gotchas.
Out of the box it wouldn’t compile, the source code was missing files, etc…
The task is to find every such hidden issue, fix it, which then allows us to find the next hidden issue, and so on.
When we’re done finding issues, we’re basically done.
Can you tell me how many issues we’ll encounter? I can’t. I’m being perpetually surprised by the code base, finding random binary blobs that actually are from some other project we didn’t even know exists until that moment.
Fundamentally, it’ll take as long as it takes. The application is mandated by law, and it must be made to work. It’ll cost what it costs, and it’ll be finished when it’s done.
That's exactly it. Scoping out the solution is 90% as hard as just doing it. By the time you've figured out enough of the details that you're sure there are no more surprises waiting... you're just about done.
I suspect that a lot of counter-examples people have witnessed are examples of poorly factored codebases. I've seen teams/projects where they estimated projects accurately because the "new" thing being created was a slight variation on a kind of thing they'd created many times.
For example, I once worked for a company that generated e-commerce sites for clients. They'd make a new one basically by copy/pasting an existing site from another client and changing some text/images here and there. Super predictable after doing this a bunch of times... but also a maintenance nightmare having dozens of slightly different codebases. And how do you iterate? They couldn't evolve.
Shopify ate their lunch. I bet Shopfiy has a really hard time estimating the time to ship a new feature. But they add it once for everybody.
I love deadlines. I love the whooshing noise they make as they go by.
- Douglas Adams
A knowledgeable developer can predict how this cycle goes and explain that an estimate of the whole task is impossible, and they will be quite literally and completely correct in that case.
Ah, that would be nice. In my experience it's more like, "Okay, I understand it's not estimable at the moment. Just make your best guess and we can revise it later. (Psych! I will hold you to this estimate like you were under oath in court!)"
This leads to things like the Scotty rule. Since management will always misuse, misunderstand, or misrepresent your estimate anyway, might as well look out for number one and inflate the estimate, just in case.
The only time I've really had management "misuse, misunderstand, or misrepresent your estimate" is once they got past the point of being several hundred people with the bureaucracy that entails. Then I leave and go to a startup that's <50 people.
In my experience good communication solves this every time.
MGMT: We need an estimate ENG: I have several unknowns MGMT: We really need an estimate ENG: This is my rough estimate, but be aware that the unknowns could change this drastically MGMT: Ok, what do you need to get a firmer estimate.
And so on.
As a project progresses, the communication between ENG and MGMT is crucial to ensure that MGMT are aware how those unknowns are affecting the project, MGMT will decide at every juncture if the project is worthwhile to continue to invest resources into it (assuming they avoid the sunk cost fallacy)
As an industry we just seem incapable of coming up with any other way of negotiating and running projects.
This would allow for planning, prioritising and is the most accurate communication we can achieve.
Because that’s what we do either way. Round numbers from ass make no science to plan and prioritize upon.
1. Assumptions (including soft such as who will actually do it, resources are not equally relevant or productive) 2. Constraints (as agreed by stakeholders) 3. Risks 4. Clear scope 3. Alternatives and why they were not preferred
Business should be aware and own the uncertainties, or else the system gets biased towards sales people offering vaporware.
HOW developers should make their estimation is a totally different question, but it is not some arcane knowledge, and it should be an everyday craft for every mid-level engineer. First of all, we are doing those tasks by hundreds through our careers, so an easy way is to remember how long similar tasks took us last 10 times an produce a bracket, like in "from 5 to 8 working days, let's say 10 to be sure". Every business person will be happy with that answer.
OP says that every engineering task is unique as a piece of art. This is BS, IMO, except for the very junior engineer. Most of the time business wants some well-known and hundred times done things they see their competitors do. All the uniqness stems from particular codebase. If your codebase is a mess - you are in trouble, but it is you problem, not the general problem of software engineering.
In case I have some really unique task, what I should do is to say: "I cannot estimate this right now. Give me two days and I will get you an estimation". In those days I decompose the task into smaller, and divide those smaller tasks into "well known" and "unknown". Then I imagine (or even draw) a trivial Gantt chart and will have my estimation.
It is also totally normal to change your estimation during the work. You just should do that the moment you see a problem, not at the day of the deadline.
Not surprised this discussion devolved into this bizarro world point so quickly. The developer says “there are too many unknowns to provide any sort of estimate”. The manager retorts “but we need estimates!” As if they expect the developer to say “oh, you need them? You didn’t say that. In that case, the unknowns are now known and it will take four days, three if you remove this inconsequential detail that nobody cares about”.
Unfortunately this bit of illogic is so common that I know it’s a waste of time to argue, so I’ve learned how to figure out what they want the estimates to be, give them that, miss them (just like everybody else does), and then get to work on actually providing the feature.