> And I can set up two micro instances with terraform in less than an hour.

The post was more of a metaphor than a concrete example.

However, as a manager I can’t count how many times my team has declared that they can do something in under a day only to have it become a part-time job for someone to maintain indefinitely, or a multi-week rabbit hole as they debug some unexpected behavior, or technical debt as they neglect to document their solution, or a productivity bottleneck as everyone else must now either learn Terraform or wait for the Terraform guy to be available to help and so on.

When I transitioned from IC engineer to engineering manager, I quickly learned that engineers frequently assume the best-case scenario for their time estimates, while managers end up planning for the worst-case scenario. The catch is that it’s difficult to tell an engineer that their time estimate could be an order of magnitude or more short of a realistic estimate without implicitly insulting their abilities in the process. ICs also quickly forget that many of us managers were once over-optimistic IC engineers as well.

> "I quickly learned that engineers frequently assume the best-case scenario for their time estimates, while managers end up planning for the worst-case scenario."

To be fair, bad managers (and there are a lot of them out there) often get really angry if you give a reasonable worst case scenario time estimate for a task, even if you can back it up.

Indeed. You learn in all roles that the only way to get your projects pushed through is to give overly ambitious timelines. If you were conservative or realistic, nothing would ever progress.

I've constantly been on projects where someone said, "This will only be a month of work for maybe 3 engineers." That project quickly becomes 5 engineers over a year because of empty promises.

That said, some of these projects frequently time a very long time because there's dishonesty in time estimates. If they were honest with estimates, you wouldn't have to cut corners. They try to cut corners to push a project through quickly - but it backfires constantly... Which leads to more work rather than doing the more effective and reliable part of lots of non-visible work before seeing visible results.

When I was managing a large, distributed team of mostly mediocre engineers (at a poor bigco), it was common to take the best case estimate (provided by a senior engineer or architect), add 50-100% because due to lack of similarly experienced engineers responsible for day-to-day coding, add 25-30% for SQA, and add 50% for "business" delays (usually LOB heads who couldn't agree on features, or who asked for utterly stupid things that we'd have to explain away before we even got started ... like a web UI with a 52 column table meant to emulate the previous Excel-based "system"). The result was frustration all around because the senior engineers knew our estimates were crazy high compared to what they'd be able to do independently, our business stakeholders were mad because of the cost, and our CIO couldn't understand why it seemed literally nothing was doable in less than several thousand person hours. Good times.

But when you 1) don't have an constructive relationship between CIO/CTO and business leaders, and 2) offshore everything not to follow-the-sun but purely to save money (and then pay bottom of the barrel even in lower cost regions -- that company still only pays engineers 10-12 lakh in south India), you're going to get an inefficient org creating crap quality software.

It sounds like you had distributed decision making authority but concentrated accountability for the outcome. Worst-case scenario.

> the senior engineers knew our estimates were crazy high compared to what they'd be able to do independently

Interestingly, I worked at a company where teams of engineers were given wide latitude to design, implement, and deliver their features in isolation. The theory was that engineers could be more productive if they were left alone to make the decisions.

I'd say it worked about 1 in 4 times. The other 3 out of 4 times, engineers created their own labyrinth of over-engineering, spent too much time building things as reusable frameworks instead of simply getting the job done, and over complicating the system with unnecessary "Wouldn't be cool if..." type features.

I find my sense of laziness keeps me from going too far down a rabbit hole most of the time.

When I was in a large organization, the upper management roadmap was based on quarters and year. Like Q1 2020 Q2 2020. That's a very large granularity of 3 months.

This had the benefits of ignoring the usual variations of software planning. Any development task could take 1 week or 4 weeks (if it goes well or not). This stops being an issue when the planning is quarterly because few weeks of jitter doesn't matter.

The question is then, do we have time to do this task next quarter, as well as the other 3 tasks we scheduled there? otherwise we'll plan it for the quarter afterward.

> engineers frequently assume the best-case scenario for their time estimates, while managers end up planning for the worst-case scenario

I (engineer) have experienced exactly the opposite. On Thursday I say "It will most likely take me a week; I cannot possibly have it done in less than three days; if X is true -- which I can't know yet -- it will be more like two weeks". And that becomes a commitment to have the code on master by next Wednesday.

Perhaps I've just been unfortunate, but most of the time when I have given qualified or worst-case scenarios, I have been pressed to assume the best, and that's what gets written down.

I want to chime in that I, also a developer, mostly experienced the opposite. I hated giving time estimates, they were demanded, and sometimes when I overshot the optimistic time, managers were incredulous.

Very few people involved understand how difficult it is to estimate work for some/most projects. If you haven't explored the problem space pretty well, you can and will run into problems that are unique and very troublesome. And you can't know that ahead of time, except in a very hand-wavy way.

> Except in my experience it’s not two years, it’s two months. And I can set up two micro instances with terraform in less than an hour.

Fucking that. Only skip the basics entirely if you don't plan to hire anyone in those 5 months (no or inadequate tests, no scripted way to create a dev instance? You just cut their first-two-months productivity in half.) and are 1000% sure no-one's gonna horrifically break prod, and don't mind dev velocity dropping ~10% (and/or bug rate and "of shit it's on fire" calls increasing alarmingly) every month over the next 6 months.

There's a lot of room between "we need a k8s cluster with autoscaling, three kinds of databases, lambdas, blah blah" and "LOL develop on prod YOLO!" IME this kind of thing never gets added in month 4 because there's already too much tech debt. Add it in month 1. A couple days and 10s of dollars/month setting up or buying tools & services to keep your team confidently moving fast over the next half-year is some of the most valuable work you'll do.

[EDIT] oh and in the specific case of Infra as Code or fully-scripted deployments, around month 4 when your stuff's a bit more complicated it makes your answer "yeah, give me these couple pieces of info and put me in touch with one of their tech guys and I'll have it done by lunch tomorrow" instead of "uhhhh... fuck... uh.... two weeks? Hopefully?" when your sales guy says "we've got a huge deal almost closed but they need our thing hosted on govcloud/on-prem/European servers, can we do that?"

At the same time, it also means collectively we don't know what we're doing to get where we're going and are just throwing stuff at a wall to see what sticks.

It also made me think of the Apollo 13 movie.

The engineering/process people got the thing into space, the adaptable/dynamic solve it with 'gaffer-tape and whatever is laying around' people got the thing home.

We still need both approaches for different reasons.

“Personal survivor bias“ accidentally chooses a side for you.

Experience eventually makes one realise there aren't diametrically opposing sides, just different capabilities needed in different circumstances.

Except the people who did the "tape it together"? Those were the same people who made the long process before. That's why they could just tape it together, they were deep in the knowledge of the system they were fixing and thus could tape a kludge together to get them back.

But that's because in those days the people that built the thing have to do production support once it goes live. That doesn't happen so much in larger organisations because they have separate teams divided along "add/build something new" and "keep it going" boundaries without a lot of cross-pollination.

There's also the difference between building bottom up with full knowledge of what you built, like the Apollo program team had (even if they bought something externally, the people who made it were available to the team, and/or the team studied the items in depth).

And then there's modern world of slapping together libraries that are often not known, sometimes with closed source, sometimes without real documentation. I've had to use disassembler in the past to access framework we were supposed to use...

Twitter had the fail whale for years. Same with reddit. Facebook moved fast and broke things. What engineers often don't understand is that engineering is not important to the business, only the preceptor of it is. Even these huge companies are only backfilling engineering on their core critical components, while new experimental projects are sloppy messes. The only reason half the stuff is solid is that it gets to free ride on comping built for the critical core.

I agree. While I'm a big proponent of "as simple as possible", often when people say "x is overkill", they mean "I don't know x" (where x != k8s of course).

I think this is 100% true. I've never used the term overkill personally.

If I don't know something, I simply say I don't know. And if I do, I always position the trade off of the choice relatively and weigh it.

Thing is, it really changes the equation of whether it's worth or not.

If you or some other senior engineer knows X, then perhaps they can implement X in an hour and things remain simple.

If your team does not know X, then it will take a week to learn X, understand it, try it out on a toy project and implement it in production (which alone might make it an overkill, because it's not worth a week) and it becomes quite plausible that we'll implement it wrong in a way that will bite us later.

So unless the benefit is huge, then it's quite reasonable to say "x is overkill" for your team simply because you don't know X.

The GP of my comment says:

> Me as a manager at a startup: "Do not install Terraform. Finish the import prototype... now."

The team knows Terraform. The manager probably does not. I was responding to that.

Well introducing some new technology that people will have to learn is often overkill, if there is already a slightly less optimal way to do it that the whole team is familiar with.

Do you know k8s?

Haha perhaps not well enough, thus proving my earlier point :-)

If you can set something up in an hour, don't discuss it; show it. Spend the hour, show it works. If it does, fantastic. If it doesn't, drop it.