News for hackers. Stuff that matters.
Re:Hard-to-swallow truths they won't tell you about software... by wccrawford on Wednesday November 08, 2023 @06:58AM (#38189228)
I've been to the store before. A lot. I've never hooked in new library X to the codebase and implemented that new functionality.

Estimating things you've never done is hard.

Estimating things you've done a lot is trivial, and pointless.

When people saying that estimating is hard, they don't mean they easy kind.

Re:Hard-to-swallow truths they won't tell you about software... by Clubber on Wednesday November 08, 2023 @07:12AM (#38189352)
"I want you to build a document management system, how long will that take?"

Uhhhhhhhhhhh....

Re:Hard-to-swallow truths they won't tell you about software... by saalweachter on Wednesday November 08, 2023 @07:55AM (#38189744)
"It will take me X weeks... to come up with a reasonably accurate estimate."
Re:Hard-to-swallow truths they won't tell you about software... by guappa on Wednesday November 08, 2023 @08:03AM (#38189811)
It will take 2 months to gather the actual requirements and be able to do an estimate.
Re:Hard-to-swallow truths they won't tell you about software... by wheresthebeef on Wednesday November 08, 2023 @08:40AM (#38190137)
This guy consults.
Re:Hard-to-swallow truths they won't tell you about software... by rawgabbit on Wednesday November 08, 2023 @11:52AM (#38193153)
From my experience, the two months of gathering requirements will be more valuable than developing something from zero requirements. Whatever you code with always be "wrong" because with zero requirements there is nothing to test against.
Re:Hard-to-swallow truths they won't tell you about software... by baq on Wednesday November 08, 2023 @08:05AM (#38189826)
the correct answer is 'yes'
Re:Hard-to-swallow truths they won't tell you about software... by speed_spread on Wednesday November 08, 2023 @08:10AM (#38189859)
Fork an existing one, slap your name and logo on it. 2 points.
Re:Hard-to-swallow truths they won't tell you about software... by locustmostest on Wednesday November 08, 2023 @08:15AM (#38189901)
Exactly. We'd love it if more people did that with our document management system. (https://github.com/formkiq/formkiq-core)
Re:Hard-to-swallow truths they won't tell you about software... by Izkata on Wednesday November 08, 2023 @10:49AM (#38192080)
"This is git. Have fun!"
Re:Hard-to-swallow truths they won't tell you about software... by orwin on Wednesday November 08, 2023 @08:02AM (#38189802)
Estimate are for accountability. It's for team management. That's why middle management shouldn't have access to estimates imho.
Re:Hard-to-swallow truths they won't tell you about software... by sixothree on Wednesday November 08, 2023 @08:30AM (#38190031)
The issue everyone is skirting here about estimates is the unknowns. Letting management know what the unknowns are and how confident you are you can resolve them is just as helpful as providing an accurate hour estimate.
Re:Hard-to-swallow truths they won't tell you about software... by lcnPylGDnU4H9OF on Wednesday November 08, 2023 @08:26AM (#38189995)
Even on the store point: I might notice I’m low on fuel and fill up on the way. Easily turns a 15-minute errand into 20 minutes. Estimations in the context of a software project are similar to a promise that my trip to the store will only be 15 minutes which I’m held to even if I notice the low fuel gauge.
Re:Hard-to-swallow truths they won't tell you about software... by ozim on Wednesday November 08, 2023 @09:31AM (#38190775)
I like the store metaphor.

Shopping list is set and I should be in and out in 15 mins but it turns out there is no ketchup - so do I go to another store to get ketchup or mustard will do. Product owner my Girlfriend has to decide because she is doing the cooking and will present it to customers our guests, and she knows if they like mustard or not because she talks to them more often than I do.

But yeah going to another store might make dinner late and I cannot know this until I actually arrive at the store that there is no ketchup on the shelves.

Re:Hard-to-swallow truths they won't tell you about software... by interactivecode on Wednesday November 08, 2023 @11:40AM (#38192918)
Its a good metaphor also because if you have an appointment in 20min you would never go to the shop 15min before. Yet managers will force you still out and to the shop because of your time estimate.
Re:Hard-to-swallow truths they won't tell you about software... by Volundr on Wednesday November 08, 2023 @12:42PM (#38194024)
Part of estimating is accounting for those contingencies. Given above, depending on my audience the above estimate will be either 30 minutes, or 15 minutes but could be as long as 30 depending on these factors. Part of estimating is accounting for the setbacks one can (reasonably) expect to see, to give the other party a timeframe within which they can be (reasonably) certain it will be done.

Of course there could be a fire on-route causing a jam that turns your 15 minute trip into 2 hours, but that's sufficiently unlikely it's probably not worth building into your estimate (but could be depending on the stakes involved! If someone is going to die you should probably factor this in so we can plan for that contingency). In that instance instead you need to communicate when it happens so that everyone involved can recalibrate.

Re:Hard-to-swallow truths they won't tell you about software... by lcnPylGDnU4H9OF on Wednesday November 08, 2023 @08:46PM (#38199952)
> the above estimate will be either 30 minutes

Yeah, I had a PM who taught me to do this by doing it themself. My estimates are actually just doubled from how much I expect it to take when I know the design and can work with no distractions.

I’ve also worked with managers who “negotiate” as others sometimes describe. Of these I see good and bad. One who was aware of short- and long-term goals of our team and asked for estimates so they could decide what features we need to include; another who already promised the features and the deadline and is asking me to tell them that they did not overpromise despite my observations. (“I don’t believe I’ll be able to deliver those features by that date.”)

Others in between, including some who don’t like being told explicitly that I double the estimate. Worse still, the one who halves it as if to correct for some mistake.

I guess the moral is that developers aren’t the only critical party when it comes to communicating estimates. Managers do their own job; good, bad, or otherwise.

-------

> depending on my audience

Other than some managers I assume another developer or a good technical manager, and it’s a good approach. Sounds at least similar to explicitly stating that you doubled your “no distractions” estimate. Too bad it seems that an unfortunate lot of people work with the bad “negotiating” type who might even be so disrespectful as to correct their co-worker’s mistaken opinion -- which they asked for!

(Not to leave this as advice for the parent commenter. Just wrote some thoughts and figure I can share for anyone who might get value from them.)

Re:Hard-to-swallow truths they won't tell you about software... by Tangurena2 on Wednesday November 08, 2023 @09:43AM (#38191012)
I think the reason that estimating is hard is that no one ever returns to their estimates and tries to fix the estimates. What usually happens is that people throw out a number, mismanagers think that number is negotiable, and whether the project's completion is in any way related to the estimate - the estimate is forgotten.

No one writes down "this is why I think the project will take X [time units]".

And no one goes back to that estimate and says "I left Y & Z out of that estimate, I better include that next time."

If you want estimates that have any relation to reality, your estimating process needs to have that sort of feedback.

> When people saying that estimating is hard...

I think they're referring to the sort where management negotiates the estimates, like you're haggling over something in the marketplace. If they have such a strong opinion about how long something should take, then asking you for estimates is setting you up for failure and blamestorming. "It’s one banana, Michael. What could it cost, $10?"

Re:Hard-to-swallow truths they won't tell you about software... by SpaceNoodled on Wednesday November 08, 2023 @11:41AM (#38192938)
I've streamlined the process, and just say that anything will take five days to complete.
Re:Hard-to-swallow truths they won't tell you about software... by sroussey on Wednesday November 08, 2023 @11:45AM (#38192996)
Teams definitely refine and alter estimates as new knowledge of the problem space is discovered. It’s a management (and team) failure if this does occur.

Also, on the negotiation side of time it will take, I absolutely know for myself that I can code things fast and functional but a bit hacky, or beautiful and elegant and will take much longer.

The negotiation should be about when to do which.

At a higher level, you put your teams together within their temperaments in mind and match that to the work.

Some code will be a nexus of changes and interaction with other code (invest early), and some code will get written once and never seen again (go fast and forget).

Re:Hard-to-swallow truths they won't tell you about software... by PH95VuimJjqBqy on Wednesday November 08, 2023 @12:36PM (#38193933)
One of the things that makes me so successful is that I always try and have a backup plan. "our goal is X, if we fail to meet that with approach Y then we'll use approach Z. Lets prepare for approach Z while we're also preparing for approach Y and evaluate and switch to approach Z as early as we possibly can. We still want approach Y so we'll continue working on it even if we start using approach Z to successfully launch".

The issue is that there's a fluidity in that thinking that many many MANY technical people either don't have or don't employ.

Re:Hard-to-swallow truths they won't tell you about software... by Joeri on Wednesday November 08, 2023 @12:43PM (#38194038)
In my experience writing out assumptions together with estimates tends to improve estimate accuracy a lot, by challenging the assumptions (and estimates) more at estimation time, and making it more visible when changes occur at implementation time that impact estimates.

As for negotiating estimates down, it is up to technical leadership to refuse to do that. Not easy, but doable.

Re:Hard-to-swallow truths they won't tell you about software... by madeofpalk on Wednesday November 08, 2023 @01:18PM (#38194556)
One of the most productive teams I worked on spent time at the end of an iteration to look back on our previous estimates and re-estimate them. We got much better at estimating the remaining work we had, and knowing whether we would hit deadlines or not.
Re:Hard-to-swallow truths they won't tell you about software... by steamer25 on Wednesday November 08, 2023 @11:34AM (#38192827)
My favorite is being asked to estimate a bug fix before I've had any chance to analyze the issue.

It's like calling a mechanic over the phone and asking how much it will take to fix your car. The mechanic doesn't know if the engine just fell out of your Model T or if you're just trying to put the key in backwards. How about you bring it in first, then we'll look it over and go from there?

Sometimes you can ask some questions and get to, "Well it's probably X which would take Y but we'll need to see". Then they get Y stuck in their head and it turns out to be the one time out of twenty when the cause isn't actually X but something much more involved.

Re:Hard-to-swallow truths they won't tell you about software... by vkou on Wednesday November 08, 2023 @01:35PM (#38194809)
If you're being asked to estimate a task without insufficient information on what the task is, punt it back with 'More information needed, we'll look at it 3 weeks from now in the next sprint planning'.
Re:Hard-to-swallow truths they won't tell you about software... by danenania on Wednesday November 08, 2023 @12:05PM (#38193377)
Accurate estimates are impossible, but you can get better at them. Getting better at them mostly entails increasing your estimate to the point that it feels absurd, and then doubling it. To become a true expert, double it again. Another approach is to always bump it up a unit of time. So an hour becomes a day, a day a week, a week a month, a month a quarter, etc.

Really, you just have to be a pessimist. Instead of saying what people want to hear (it's simple and can be implemented quickly), you need to imagine the long-tail worst-case scenario where everything that can possibly go wrong goes wrong, then give that number. If people don't give you a somewhat incredulous look when you give an estimate, you probably were too optimistic. Just remind them that you're imagining a worst-case scenario and say that you hope to have it done sooner, but don't back down on your estimate.

Another thing to realize: at the business level, deadlines are often picked arbitrarily to create urgency. The specific date of the deadline is usually less important than just having some deadline, any deadline, to motivate everyone. Business people will pretend the deadline is Very Important, but it's all an act, more or less (arguably it's a necessary one). Remember this when giving your estimates.

Re:Hard-to-swallow truths they won't tell you about software... by PH95VuimJjqBqy on Wednesday November 08, 2023 @12:33PM (#38193880)
> Accurate estimates are impossible

disagree, not all tasks can be estimated accurately depending on circumstances but this does not imply that it's impossible to estimate any task accurately.

Re:Hard-to-swallow truths they won't tell you about software... by danenania on Wednesday November 08, 2023 @12:50PM (#38194139)
No matter what the task is, things can go wrong that you didn’t anticipate.

Maybe it’s just a quick CSS change, but all of a sudden the webpack build starts mysteriously failing. And now your node_modules got corrupted and oops, when you delete it and reinstall dependencies, there’s a dependency conflict. Once you’ve finally gotten that figured out, some tests are now failing in part of the codebase you’ve never seen before. Better make some coffee…

Some version of this can always happen, and as a task gets larger, it is almost guaranteed to happen multiple times in multiple variations. Since you don’t know what problems will arise, there is no way to know how long it will take you to fix them. You can stay up all night in order to pretend that your “quick and easy” estimate was accurate, but that only takes you so far.

Re:Hard-to-swallow truths they won't tell you about software... by PH95VuimJjqBqy on Wednesday November 08, 2023 @01:17PM (#38194537)
yeah, and maybe I'll have a heart attack and die while going to take a piss but that doesn't imply it's impossible to accurately estimate that it's going to take 5 minutes for said piss.

What if, while cooking dinner, a meteor crashes in my kitchen and so my estimate of 30 minutes to cook dinner turns out to be incorrect? That _obviously_ means the conclusion is that it's impossible to accurately estimate how long it takes to cook that dish so we should never do so.

There has to be a name for the type of reply your post represents.

Re:Hard-to-swallow truths they won't tell you about software... by OkayPhysicist on Wednesday November 08, 2023 @01:39PM (#38194874)
Software has an unusually fat long tail, to an even worse extent than real engineering. The probability of something unexpected popping up and inflating the time something takes by an order of magnitude is much, much higher than the likelihood of the same happening while cooking.

This ends up coupled with the fact that practitioners (of anything productive) tend to think in terms of 50-ish percentile estimates (i.e., there's a 50% chance this project is done in this amount of time), whereas business people care about 90th-99th percentile estimates. In software, the difference between between a 50th percentile estimate and 90th percentile estimate can be more than an order of magnitude in time spent.

Re:Hard-to-swallow truths they won't tell you about software... by danenania on Wednesday November 08, 2023 @01:52PM (#38195092)
Well said. 50th percentile vs. 90-99th percentile estimates is a great way to put it.
Re:Hard-to-swallow truths they won't tell you about software... by PH95VuimJjqBqy on Wednesday November 08, 2023 @03:49PM (#38196748)
I'm going to repeat what I said to another poster.

If your shit is breaking so often it's affecting your willingness to estimate, that's a you problem.

I'm driving to another state for the thanksgiving holidays, and I have an estimate for how long that will take. Is it possible when I walk out to my car that the battery died the night before? sure. Am I going to factor such an unlikely event into my estimate? no. Does that imply it's impossible for me to accurately estimate how long that trip is going to take? No it doesn't.

Only in software could someone be using a tool that constantly doesn't work and they never consider replacing it or fixing it. Imagine if a carpenter showed up to your house with a hammer whose handle fell off once or twice an hour but they expected you to pay them for the loss of productivity rather than purchasing a hammer whose handle doesn't fall off.

Re:Hard-to-swallow truths they won't tell you about software... by OkayPhysicist on Wednesday November 08, 2023 @04:24PM (#38197283)
Not quite sure how you found your way onto a tech forum without any understanding of what software development entails, but unlike carpentry, where every time you want a new wall the carpenter needs to go through the same process to make the wall, or in cooking, where every time I want a roasted turkey the cook needs to cook a turkey, software almost never does the same thing twice. If you find yourself doing the same thing twice, that's a tool problem.

That produces a much fatter tail distribution, because you're doing someone that no one has ever done before, and thus no one knows exactly how long it's going to take.

Re:Hard-to-swallow truths they won't tell you about software... by PH95VuimJjqBqy on Wednesday November 08, 2023 @05:15PM (#38197919)
> software almost never does the same thing twice

odd, because my experience is that your CI/CD pipeline does the exact same thing repeatedly.

Perhaps the answer to your confusion about my experience is that I don't suffer tools that hamper my productivity.

Put it another way.

I ran a raiding guild for several years and about halfway through that stint I recruited a player that raised the bar for what I thought was possible. Perhaps you need your bar raised.

Re:Hard-to-swallow truths they won't tell you about software... by danenania on Wednesday November 08, 2023 @01:24PM (#38194655)
Haha yes, webpack breaking or a test failing is equivalent to having a heart attack or a meteor hitting your kitchen.
Re:Hard-to-swallow truths they won't tell you about software... by PH95VuimJjqBqy on Wednesday November 08, 2023 @01:45PM (#38194988)
if your webpack or ci/cd pipeline is breaking often enough that you worry about it when delivering estimates, that's a you problem.
Re:Hard-to-swallow truths they won't tell you about software... by danenania on Wednesday November 08, 2023 @02:03PM (#38195254)
Are free ponies available in this magical land you speak of where nothing ever unexpectedly breaks? Please put me down for two.
Re:Hard-to-swallow truths they won't tell you about software... by PH95VuimJjqBqy on Wednesday November 08, 2023 @03:39PM (#38196618)
you mean the same world where no one has heart attacks and meteors have never hit houses?

Both have been documented to have happened, yet somehow you didn't seem to think that should be taken into account when considering estimates for cooking and pissing.

If your shit is breaking so often you're afraid to give an estimate, that's a you problem.

And yes, we had an internal team that w/i the last 6 months couldn't go a week without a deployment problem. Ask me why they no longer have that issue.

Re:Hard-to-swallow truths they won't tell you about software... by arcanemachiner on Wednesday November 08, 2023 @04:55PM (#38197665)
> Maybe it’s just a quick CSS change, but all of a sudden the webpack build starts mysteriously failing. And now your node_modules got corrupted and oops, when you delete it and reinstall dependencies, there’s a dependency conflict. Once you’ve finally gotten that figured out, some tests are now failing in part of the codebase you’ve never seen before.

This is too real.

Re:Hard-to-swallow truths they won't tell you about software... by NoGravitas on Wednesday November 08, 2023 @04:10PM (#38197071)
[Managers] are like children. They want everything right now, and they want it their way. But the secret is to give them only what they need, not what they want. — Montgomery Scott
Re:Hard-to-swallow truths they won't tell you about software... by grecy on Wednesday November 08, 2023 @01:32PM (#38194768)
> I've been to the store before. A lot. I've never hooked in new library X to the codebase and implemented that new functionality.

Which is why you'll get better at estimating as you get more years of experience under your belt, and why estimating is something that seniors are better at than juniors.

Even after 20 years you may never have hooked in new library X, but I'd be willing to bet you've done something kind-of similar, and your estimate will be more accurate than a person with 1 year of experience and has never done anything remotely similar.

> Estimating things you've done a lot is trivial, and pointless.

Keep in mind you are not providing the estimate for the person who has done the task many times, you're providing the estimate to people who have never done it before. So for them, it absolutely is not trivial.