This has been the ideal theory for decades and I really wish people finally stopped to think for two minutes, look around, read some history and finally understand that it's not working.
Is it really so hard to accept this reality? I am periodically mind-boggled by the amazingly resilient illusion that you can plan things as tidily. It never happened, not once, in my 20 years of career.
Estimating these costs upfront is more or less impossible. It's usually a can of worms that opens other brand new and exciting sub-tasks, the same kind that gives managers nightmares.
IMO you might have projected a little bit and arguing against some imaginary "us the programmers are better than everyone else". Meh. Only kiddos think and say things like that. By and large, almost all experienced programmers I've ever met are quite moderate and humble.
Problem is, they are too moderate and humble which leads to predatory businessmen micro-managing them to oblivion. I've personally witnessed high-profile programmers firmly saying "No!", putting their foot down and making a thing their way with more time budget. Their project and code was used, almost untouched, for 10+ years afterwards -- it was that good.
Many of us are expert artisans. You don't go to a professional blacksmith giving him wisdom how can he accelerate a certain process.
I recently made the mistake of deferring to the anxieties and fears of non-developers for so long that I almost forgot how we got to where we are today or what my true capabilities are.
You know exactly what those assholes are thinking when you say the word "refactor" or "iterate". It starts to become a psychological horror experience if you let it get out of control for too long.
Now don't get me wrong. At the end of the day I love and care for our customer too. I just don't let their minute-by-minute feature request stream live rent free in my head the same way the rest of the team does.
Ouch, dude, you didn't have to stab me in the heart. But yeah, 100% same. I am just now, at the age of almost 42, starting to finally crawl out of that. I am ashamed that it took me most of my life and career but oh well.
> I think the key point for me is that once you control for all common human flaws, the developer is still far better suited to make an executive decision regarding a technology concern than anyone else on the team.
True, but then you get blamed for "not understanding business realities". I got super tired of being hinted that I "don't get it" or whatever, especially having in mind that I launched products whose launch date I was not consulted about, repeatedly, and still get treated like dirt. I started pushing back on micro-management, fiercely and sometimes rudely.
You know what's the funniest part? It works. People start looking at you very differently and suddenly negotiations are an option. I guess it was possible all along and the business people are like "push him, he's a tech nerd, he'll fold".
We should start getting aware how are others viewing us and integrate this knowledge into our negotiation strategy.
I don't think either devlopers or managers can estimate future savings in most cases, but I still think it's necessary to refactor just to not drown in complexity and slow down overall development speed. My approach is to reserve about 20% for refactoring and technical improvements and let the team decide internally what to use it on.
Enough to get useful stuff done, small enough to keep most capacity for feature development. Also depends on the amount of technical tickets deemed relevant by the team
I think I just fundamentally disagree that this is the managers job. He tells me what needs doing, and I get that done as soon as possible within a given block of time (mostly sprints, so two weeks). They should be able to trust me to refactor as necessary within that block.
I think the more interesting scenario is large tech debts that take real time, like the change itself is weeks, or months. These need to be justified and planned and coordinated with managers.
In terms of refactoring there is no way to give a Cost Benefit analysis as that's too subjective. This should be a maintenance cost that's factored into the project at a high level. So you would say there a X days per year (normally about 20%) of the working days of the year spent on maintenance. Company I'm at has this and if you're doing this type of work then you have timecodes to book your time against.
That includes refactoring, re-architecture, tech debt, updating deps ...etc. PM's can't override this for their own project i.e. they have to allow for 20% off project activities in there Project Plan. Devs shouldn't spend more that 20% of their time doing these activities over the FY.
Example here would be in the UK we have something called a MOT which you have to legally get it's like a minimal maintenance you have to get on your car. Most people spend a little more though than the minimum.
Ideally, yes, though in practice this usually breaks in various ways, especially when problems occur. E.g. the President of a country probably doesn't need to be a firefighter, but if there's suddenly a massive fire, I assume he'd need to be briefed in far more detail to help call shots around the type of response.
> In terms of refactoring there is no way to give a Cost Benefit analysis as that's too subjective.
I mean, you probably won't get a very correct Cost Benefit analysis, but you have to do something. The rest of your post is you describing that something - if you elect to spend 20% of your time on maintenance, that's effectively giving it a certain amount of weight compared to other things you can be doing with your time. That's certainly one way to work.
Personally, I think a more flexible approach is better. E.g. new startup-ish projects should usually spend a lot less time refactoring, for multiple reasons, whereas more established long-term projects should spend much closer to 20%, or maybe more.
I feel like this is something a lot of presidents (or VP’s, or Directors, etc.) think, but really, instead of taking time away from the professionals doing the fighting to explain to them the minutiae of blowback, the best thing they can do is say “I trust you, put it out.” And then leave the pros to do their job.
In other cases, it is more important to get things right and continual refactoring is the fastest way to find an acceptable architecture to a novel problem.
In either approach, when speed is of essence, a limited scope helps avoid either producing insurmountable technical debt, or an unterminating research rabbit hole, respectively.
In the case I gave it's hours book on the maintenance project that is the test but you could very well have other measures and I've seen these in many projects. Things like linters, dependency bots, unit tests, e2e tests, code coverage, Cyclomatic complexity ...etc.
But the way the conversation should go isn't the developer deciding when refactoring should happen. It's the developer giving an analysis of the cost and benefit of a refactoring (it will take X time, but will save Y work in the future). And the manager factoring that into all the other circumstances, and deciding whether it's worth the current cost.