There were many times where I refactored code that never received any additional updates or to have the feature removed. I wasted a lot of time without any real gain for it.
There are real costs to paying off tech debt and it is easy for engineers to spend their time on low value refactoring.
Software engineers are not always good at prioritizing the business value side of their work so they aren't trusted there.
If you can show you can prioritize business value then it is easier to convince management to give you two weeks to clean up tech debt.
Since we're throwing out anecdotes, I've lost count of the number of companies I've seen that were essentially zombies due to technical debt. They had aging, rotting codebases that were impossible to add features to at the rate needed to continue making sales. No senior engineers who could improve the situation wanted to set their careers back by working for them.
So whenever you consider fixing technical debt, ask yourself if the team is skilled enough to actually create a cleaner solution. If they can't then it is just a waste of time. The best predictor of this is that they have created a cleaner solution elsewhere, if they haven't then don't let them work on technical debt unless they have an extremely well thought out plan.
Ironically in big corporations I think that this actually plays into their underlying desires - giving a boost to headcount gives them more importance and having more developers means that the leverage each engineer has is commensurately less.
I've worked on more than a few projects which would have proceeded 4x faster with 2 good developers on a clean code base than the 7 they actually had.
Indeed the entire software startup industry is probably reliant upon this dynamic.
Uber is one of the outliers in this regard. But if you look at most successful others (Instagram, Whatsapp and a bunch of others I don't want to name since I want to stay anonymous) all had very small teams right until they were acquired.
I work in a non-abstracted code base, but all that really means is every time someone wanted to do something they simply wrote their own way of doing it. Needless to say it's suboptimal to work in on the best of days.
People like to deride one way of doing things vs another but in reality pretty much every approach to software development and architecture can be amazing or hellish depending on the aggregate quality of the developers, their management and the lines of communication between them.
TL;DR - Bad code is bad no matter the paradigm and all paradigms can be made awful.
This.
Software engineers are focused on the implementation side of things and how today's work will impact their work in the future. Yet, sometimes a quick and dirty solution that takes close to nothing to implement is more than enough to put a functionality in production. Sure, the whole thing will need to be scrapped in the near future when rewriting the feature to meet all design requirements, naturally the quick and dirty solution underperforms, and of course implementing it properly will take 3 or 5x the time. Isn't it obvious that the team is wasting it's time by implementing a quick and dirty solution?
Yet, the quick and dirty feature does shorten the time to market to a fraction of what it would otherwise take to implement properly, and perhaps that is good enough for the project for the time being.
There are exceptions, of course, and I'm pretty happy to have worked with them, but in general about 90% of managers I've worked with have resisted my attempts to get them to be explicit about the level of quality that they want.
When this happens then the devteam's negotiating position changes completely, because they are no longer arguing about how to allocate resources to deliver a functionality and instead they are asking to waste time rewriting something that was was already delivered and is supposed to be up and running.
Very few people seem to actually believe this in their bones. All talk about being "agile" and "pragmatic" becomes a joke when it's posed as somehow being in contradiction to design and anything that goes deeper than finding a place to insert your new if statement.
If your management needs 'convincing' then there's a problem.
Either the manager is technical enough to know whether a particular set of work is the right thing to do or not, in which case they don't need convincing. Or they're not, in which case they should defer to the engineer and also not need convincing.
The only managers who need 'convincing' on technical matters are technically inept individuals who nonetheless attempt to micromanage technical decision making. They are no more able to link business value to technical decisions than the engineers you say you distrust. Fire them.
I disagree. Most engineers I work with like refactoring. It’s usually well defined work with clear metrics for success. It can also reduce daily pain if it makes delicate/messy code much easier to work with.
The problem with refactoring is that its only value is in improved execution of future projects. So it can be extremely valuable or completely useless. It depends on how likely the refactored things are to be worked on again.
I think software engineers tend to overestimate how prestigious and valuable people outside of tech think software engineers are.
I work in computational biology, and even in a fairly tech related field, you wouldn’t believe how many people think software engineering is just glorified IT. I’ve had a lot of conversations where the crux of it is “why should we pay them so much money, it’s the computers that are doing the work. Or, on building a distributed computation engine to run for 2 years straight without failing: “it’s all just grunt work we’ll just get an undergrad to do it”.
Why should we pay the blacksmith so much money? It's the armor doing all the work. Why should we pay the stone mason so much money? It's the arch that's doing all the work.
Yeah.
I think, sadly, this is mostly on software developers though. Look at all the pop culture stereotypes of programmers: whether those stereotypes are true or not, that is what culture thinks of us. Would you take seriously what a big-bang theory character would say? The problem is, I don't see most software engineers valuing social skills to better relate to other professions. Instead, there's a lot of whining about being misunderstood. And we are misunderstood! But man we're not helping it at all.
I don't even think that 10% really needs to change. We need the RMS' and Torvalds of the world. But we also need to, individually, act better than they do and advocate for ourselves as adults.
I constantly see coders deriding any decision that places the business over their own individual interests, and then we get indignant that the decision makers don't listen to us. But if we don't care about their interests, and the collective best interest of the company, why should they care about our interests?
“I don’t care how you do it, increase sales by 20% in Q4” creates the sales and marketing version of tech debt. (Channel stuffing, excess discounting, etc)
A few friends of mine work in accredited professions, and it's not something they have to deal with all that much, most likely because they are all accredited. They are billed by the hour, and typically if someone isn't happy with the cost the response is "okay, we'll down tools". Inversely, companies that bid on contracts will bid with a fixed cost in mind, and when estimates are taken that go over the requirements things will be fudged to fit the requirement in for that price. This means one of three things: corners will be cut, developers have to work faster, or requirements are cut. In my experience, the last point is a rarity.
I know that we software engineers like to think of ourselves as thought workers, and on the same level as the likes of lawyers and chartered accountants, but the way our industries operate are wildly different. I prefer to compare software engineering to a trade like carpentry, plumbing, etc. We share many similarities, such as easy barrier to entry and working on fixed-cost projects, but unlike trades, we're always willing to negotiate on our next pay day. You can say it's down to a difference in costs, but even on an industrial level where the person doing the sale isn't a trades-person, if a job requires £x and the client can only pay £y, then the requirements change or the job isn't taken on.
I also can't say we aren't collectively to blame, or at least somewhat - as a bunch we can be poor at political negotiation/skill, lack of standardization, etc. I kick myself every time I miss an estimate that I've given, because I know at some level that has to erode trust. Or when my manager does press for more details as to why something should be done and I can't come up with a really strong justification at that moment, etc.
Physicians enjoy a similar problem. Everyone goes to Dr. Google an then brings their misunderstandings to the clinic. Anything goes wrong, it couldn't possibly have been the fact that the patient was a brittle diabetic with hypertension. It must be the care they received in the hospital after they had a heart attack while driving and ran into a tree. Nor is anyone else in the hospital possibly making mistakes. Clearly it's the board-certified physician who doesn't know what they're doing.
I’ve been watching a series called ‘Mystery Diagnosis’ about people with rare diseases and how they were eventually diagnosed. Something you hear most of them say is along the lines of ‘Finally I met this doctor that actually listened to everything I had to say’. It’s a really interesting series — and damn does it remind you how lucky you are to be mostly healthy!
McKinsey did a study recently which found that large IT projects ran on average 45 percent over budget and delivered 56 percent less value than expected [1].
An executive is interacting with people from multiple industries or business functions and this is on the high side. It's much worse than a function like accounting, and it's also worse than many industries with bad reputations, like construction. So over time they are going to come to regard estimates from the tech guys with more suspicion than most. To the extent that other industries have this problem, yes you will find similar trust issues.
If you are a PM/product manager then the best way to handle technical debt is to focus on the benefits that come from paying it down. You are not scheduling time for technical debt, you are scheduling time to improve reliability, security, scalability, etc. and from day one you are lobbying for more time to do every feature so that it's done right in the first place. Yes, sometimes one of these things has to go wrong before the business people start listening, but if you were calling it out from day one they will respect your input that much more.
If you are a developer it's important to understand what work has business value and what doesn't, and make or inform decisions appropriately. Frankly there are developers out there whose attitude amounts to "Your job is to pay my salary, my job is to make whatever technical decisions I want without oversight, and if we don't deliver on time/on budget it's not my problem."
Not all tech debt is worth paying down, the decision generally should be made based on the "interest rate" of the debt (how hard is this going to burn us in the long run?) and the term length of the "loan" (will this component be around for one year or ten?).
[1] https://www.mckinsey.com/business-functions/mckinsey-digital...
Every time I've ever seen the approach of "create a business case for each refactoring" put into practice it has made the problem spectacularly worse because the developers are reluctant to manufacture sales pitches for things where the payoff is hard to recognize.
Hell, developers are reluctant to manufacture sales pitches period.
They're typically awful salespeople. If they're good at making business cases, and few are, they're probably not working for you. Making the quality of your product contingent on the skills of your developers at creating sales pitches is a brain dead own goal.
The best approach is to hire good engineers, allocate a rough proportion of their time that proxies the level of quality you want (e.g. 30%) and trust them - let them figure out how to allocate that time. Coz if you can't trust them you're probably fucked anyway.
>If you are a developer it's important to understand what work has business value and what doesn't, and make or inform decisions appropriately. Frankly there are developers out there whose attitude amounts to "Your job is to pay my salary, my job is to make whatever technical decisions I want without oversight, and if we don't deliver on time/on budget it's not my problem."
I can't begin to explain how incredibly damaging this attitude is. It's the philosophy of a micromanager.
Seriously, anybody who wants technical oversight of developer decisions should either become a developer or they should fuck off.
Is that fault of IT people or rather did business people over-promised? Because my repeated experience is that we give estimates, they are considered big and shorter estimates are sold. And it has nothing to do with IT people over-promising. I have yet to see a team where it is tech people pushing for shorter deadline and management being like "yeah, it is too risky, lets go with bigger estimate".
There are situations of me making short estimate, but I also make estimates too big in other situations. However, somehow the estimate that makes its way toward contract is always short due to negotiating pressure.
PM are all too often non-technical, it is quite unfair to blame "IT people" on mistakes of general management and project management. Project management blaming the these contracts being basically frauds on anyone else then themselves is literally them blaming scapegoats for their own faults.
Another problem is that especially in large organizations there are people who are literally trying to scam their way into money. You always have people aggrandizing their work, or belittling others. Especially as companies become profitable, they attract people who are only out for their cut. There are people who do nothing but attend meetings and try to scoop money into their pockets. This is the norm.
Now marry the two issues and you can see the result: someone says they have to do "special work" that will delay the project and that they are actually saving everybody else's asses. Nobody that isn't involved in doing the work can evaluate the claim. Are they just avoiding work/blame? How do you know?
I find that usually in organisations I work with, there is trust up to the D-level manager (i.e. those who directly manager developers), but there is often very little trust from other parts of the organisation (sales, upper management, or sometimes even program managers).
The best way to engender trust is to work as tranparently as possible. In the best of worlds you would deliver something every day and it would be obvious what's in the queue. Things would be in the queue for a very short time (say 2 weeks max). Work like refactoring, writing tests, improving build systems, etc would be just things you do every day and nobody would ever mention it. Of course, not everybody lives with an ideal situation, but I think most teams should strive to move their situation as close to this as they can (there are definite exceptions, though).
When each component is roughly one week's worth of code, it becomes easy to delete a component with a model that no longer fits the problem domain, and rewrite a new component with a new more appropriate solution.
"If you can rewrite a piece of code in a week, you can understand it in a week." - Greg Young
https://maximilianhoffmann.com/talks/programming/the-art-of-...
But I agree, it's definitely not easy to keep things properly decoupled. Requires some level of discipline or some savvy choices up front in the overall architecture of the whole app..
It's kind of like we're asking for it.
In SD we have people with 3 people under them who are clueless. There is no construction company in the world where a guy who has 3 people under him doesn't understand the actual work being done.
No wonder there is a trust issue when there are such huge knowledge gaps, it's not uncommon that someone's first direct superior already understands very little about what they do.
Although I have a feeling that this issue is not present at "top" companies. I have a suspicion that at a company like Microsoft, you'd have to go quite a few levels up the chain before you get to someone who is technically clueless. I doubt they'd hire some guy who never wrote a line of code and have him lead the Visual Studio team or whatever. This is something shitty companies do, it's not an unavoidable fact of life.
Basically there should never be a situation where someone's job is to write code, but their boss can't understand that code. The knowledge gap between boss and employee is too large. A boss always needs to be able to understand the work their employee is doing. If you're at a higher management level where your direct reports do not write code, then sure, you don't need to understand code.
As for the distrust, generally there is can be a lot of distrust in things you don't fully understand. Legal I think is a great example. There are a lot of contracts that screw people over and legalese is so hard to read, I have a hard time trusting what is going on. But one big difference is it is very difficult to go back and change legal terms. I was changing an LLC to a c-corp in order to receive investment and it took 3 months to fix the "legal debt" I had incurred by starting with an LLC with outside parties who had a say in what the new terms were. And it's hard to say whether we did all the right moves in that transition because I'm not a legal expert. Just as some people doubt the decisions I make in software.
There are very few professions that have as much leeway as we do to begin with.
Yes, any domain which doesn't produce a product, conceivable to a layman in one way or another. You need conceivable results to be understood by managers and other people. For example domains producing tangible results are doing much better.
I used to be electronic engineer, and it was way simpler to explain laymen around you that you actually doing something. Blueprints, boards are conceivable more or less. Lines of code are not, especially hard to explain that less lines of code is better, since laymen don't see architecture, they see just text.
A good comparison might be (building) architects: you know that if you let an architect run wild, there's a good chance you'll end up with something that is beautiful but wildly impractical and over budget.
(And to a lesser extent, every expert wants to do a good job, not a good enough job.)
LOL, seriously? Guess you haven't worked that much with other practitioners, for example lawyers.
There is some distrust always with business relationships, and I would say that it makes sense not to trust everyone 100% all the time. In the opposite, better not to trust everyone fully, even yourself.
If you want to be trusted start acting like normal people. Stop talking in technical quibberish. Stop behaving like a nerd. Shape up. Get some social interaction with normal people. Stop complaining all day long and start delivering what was asked on time and on budget.
Do people in other professions have the same issues? If the salespeople say this thing must be done in this particular way, or if the lawyers say this thing must be done in this particular way, are they questioned and overridden by their managers? I honestly don't know; it feels like software engineers are given a special level of distrust but it just might be that I'm perceiving that because I haven't experienced the other jobs.
FWIW where I currently work, we have an amazing middle-manager / product owner who has to fight the same battles with upper management with regards to process improvement and technical debt. She gets it, but has to fight the same battles we do. Sales and C-level want everything now with as little investment as possible.... I wonder if I just answered my own question?