No matter what you do, there will be tech debt of some sort. There's no perfect solution when you unpack a problem.
My metaphor for this is a stocked product warehouse. Organizing the shelves of product and keeping it that way is obviously the right solution so people dont have to hunt when pulling. That requires someone to maintain (typically the floor manager). That's not a random schmuck you can hire for min wage to do (oddly enough). You now have a form of debt requiring you to vet a proper individual to do that job if its opened up (time spent finding someone in a smaller pool that typical temp unskilled labor). Next you have pulling with forklifts. You're supposed to hire certed folks for the job, thus a skilled worker. They cost a bit more than people who can only toss crates. But the trade off is moving pulled pallets faster. Thus you have a higher per worker cost, but higher efficiency.
Tech debt is what you're willing to trade off. Highly abstracted code in an esoteric language while super efficient and well maintained requires vetting the right devs from a very small labor pool. You lose too many of those, your business can suffer greatly if you can't fill those spots quickly. Does the project require that? Can you get away with common, off the self libraries in a mass market language and structure the code where you can take low skilled devs and ramp them up quickly or ideally need no ramp up to continue working? Over engineering, no matter how well done and beautiful is not always the answer. Like, you dont want an Formula 1 designer making everyday sedans.
Beating that dead horse, tech debt is what you're willing to put up with depending on your circumstances. Can it roadblock you? Sure. But that's just piss poor planning or forgetting the original purpose. A lot of times when tech debt is brought up, it's like the person is leaving the possibility open to place a Falcon 9 rocket on their car. It may not be that the software architecture is wrong, sometimes the idea is just stupid.
I’d like to add a caveat though. At least the places I’ve worked, tech debt is a polite and convenient way to provide political cover to all individuals.
“We made some dumb choices and I don’t want to point fingers but the more we let this go the more exponentially time consuming to maintain this will become,” and/or “some of our team can’t figure out how any of this works and we don’t want to say those people are _dumb_ exactly, but working this piece of the application could improve our ability to make modifications and generally improve the competency of our team” or maybe even “this star engineer gets obsessed with this code-shape and we don’t want to lose her, so we should let her write it ‘the right way’ to keep her happy.”
These are examples were “tech debt” is being used not necessarily for lack of understanding about the problem, but rather as a shorthand to discuss a *mostly already understood problem” while providing political cover to everyone involved. Maybe management insisted on some dumb things, maybe engineering did some dumb things, maybe if we don’t do this dumb over-engineering task we’re just going to be distracted by it for the next year.
When the “tech debt” is NOT a clear trade off being discussed using those words as a shorthand, I think it’s that there’s a lot of little issues that can come back to haunt us in a wide variety of ways that we already know how to fix (eg security, documentation, efficiency of a core process could sink us if we have unexpectedly good success, whatever, and general “risk reduction” activity that is a million little things that are too plentiful to identify in isolation with any amount of efficiency). Calling it “tech debt” in this scenario is an admission that we don’t understand the risk profile of our own code well enough to make good choices, so we’d like to spend some time organizing the code so that we 1) can understand it will enough to maintain and modify and 2) better understand the potential risks of our code (and potentially mitigate some risks along the way).
It’s sort of like, you’ve got a bunch of floor managers, but you had them all just tossing product into the warehouse as fast as possible rather than doing it in an organized way, and now they’d like to maybe do their job so that we can actually have an inventory and get what we need when we need it. Yeah, you hired floor managers, but it doesn’t magically get organized if you reprioritize them to shove everything in in whatever way is fastest. Ie, it was a dumb idea to do it that way, and maybe some of that dumb is on upper management for telling the floor managers to “just shove it in ASAP” and maybe some of that dumb is on the floor managers for not confronting upper management about it before starting, but now that we’re here, let’s just agree that there was some dumb that happened.
On point 2, you're right they should be separate, but I think the solution is, refactor to make your solution easy, then build the solution.
Point 1, if you have a significant rearchitecture to mention on your roadmap, do so. But don't make the mistake of thinking business leadership cares. You'll get farther with strong technical leaders who can push for appropriate timelines, not haggle over technical tradeoffs with executives.
Parent post is terrific and makes a lot of great points.
If you work in entprise and find bad code that's just plain old vanilla bad engineering and we should carve that out of the definition.
If you work in early stage or barely mature then the luxury of defining and scoping tech debt is something that should be treated as a necessary step to maturity. After all, whatever you think of the code or design decisions you've found, it has clearly justified itself just on basis that the company is successful enough for you to be there finding it.
I've almost never seen a situation where intentionally bad code has helped even in the short term. Maybe bad code was the best that a particular engineer could do, and nobody else was available to do it, so that made it acceptable. But any engineer writing code less well than they are able to has never led to anything good in my experience.
Even in 45 minute interviews, if I take shortcuts, I usually end up creating more problems for myself than if I had just written code the right way in the first place.
Don’t kid yourself. That is technical debt, it was our fault.
Technical debt isn’t “bad code” it’s what stops you from shipping features; and it’s more important for startups to manage, not less.
What youre referring to is very early stage stuff, where the total LoC is so trivial that a few people can just remember all the details of it in their heads, so it doesn’t matter how messy it is.
...true, but there are finite physical limits to humans; you can only keep doing that for so long before you flub it up, and lose a lot of money for your employer.
This is one of those moments that people just flat out fail to have intuition about.
Same process, same results right?
Nope.
Same process with diminishing returns <— reality of software engineering over time.
So specifically:
> After all, whatever you think of the code or design decisions you've found, it has clearly justified itself just on basis that the company is successful enough for you to be there finding it.
This is wrong.
It was the correct decision/code at the time for the scale at the time, which, since you are there looking at it, it is probably no longer suitable for.
Not intuitive is it?
Software unlike physical stuff doesn't rot/depreciate, the world instead changes around it while it stays constant. Business context and requirements change around it changes to the point where the software itself isn't as useful anymore and it has to be contorted to do something it was never designed to do over time. That's it.
In tech where things the world can move fast this is particularly true.
For example one bit of advice I've heard is that things like unit tests and a CI pipeline are a luxury for early-stage startups. An experienced developer would probably spend a day or two setting things up with, say, Heroku and a Gitlab pipeline, and at least write some unit tests for core business functionality and a few tripwire tests for the HTTP endpoints. They've done the same thing a few times and could probably just pull in a starter template for 80% of that work. Very little time spent with zero impact on velocity but lays a good foundation for later.
Another junior dev tell is getting power hungry with the freedom of a greenfield project. For example, rather than going with the boring but stable PHP framework they used at the last company, they want to do all the cool things like serverless or JS framework du jour or Kubernetes or GraphQL, depending on whatever Medium or Hacker News post they last read. An experienced dev will use "boring" technology they are familiar with as much as possible, and adopt or write new tech when there is an explicit need. For example, if your startup's secret sauce is an ML algorithm connected to an HTTP API and web app, you want to focus on that ML algorithm and tweaking and optimizing that, not figuring out how to get authentication to work with your serverless/graphql/svelte cool stack.
Making code changes means one takes on the risk of breaking things. I think the people who don't break stuff are the ones who continue to refactor. The ones who don't either are scared of breaking stuff or had a bad experience with breaking stuff.
I suspect the people who refactor have what can be describe as perfectionism or obsessive compulsive.
I'm in that bucket. I can see certain code which is too complex or didn't handle a requirement well and I want to get it to be stated more simply.
You either didn't fully read my post, or didn't understand it. It's not that you can't, it's that it's a bad idea. I would instantly reject any pull request that wasn't compartmentalized to the issue at hand. It's an organizational nightmare to handle both features as well as refactoring in the same PR.
But obviously to each their own.
What would happen if you said you need another developer to hit that delivery goal. Good chance you'd get it, right? A good EM should be able to articulate the value from non-functional work and refactoring.
The former might be a big deal for a junior but can easily be paid off with pocket money by a senior (refactored on the go). While the later more naturally should be part of roadmap and business decisions, even for a senior.
To clarify the different levels, first could be something trivial as not following naming standards, missing unit-test or using global variables instead of DI, and the bigger debts would be whole stack depending on some vendor cloud lock-in.
Both of those can be anchors that can assure quality and coherence, but they can also become anchors that prevent the ship from moving.
Documentation is a huge pain for me. I hate writing it (but I'm actually fairly good at it). Code documentation is something that I've inculcated into my personal process[0], but "product docs" can be a whole different story (not just things like PDFs, but also support sites, etc.).
The biggest problem with both documentation and unit test suites, is that they can easily become "concrete galoshes"[1], that add a great deal of work to a lot of product modification and improvement strategies.
Can we call something that prevents or makes extension and modification more difficult "tech debt"? Maybe, maybe not.
It's definitely possible to write unit tests in a fashion that helps to modify codebases, but that takes a fair bit of planning and care; not something that our industry is known for, when we are in a "ship crunch" (I have known many organizations that deliberately try to maintain a constant state of "ship crunch," because their managers believe it to be more productive). I tend to prefer test harnesses, over unit tests, because of this[2]. In fact, just this morning, I was lamenting that I need to make some changes to a heavily unit-tested codebase that may cause issues with the unit tests. The fixes need to happen, even if they pooch the tests, but I will need to go back and re-run the tests, and fix any that erroneously fail. It's a really big job, for this codebase. My testing code is generally much larger than the code under test.
[0] https://littlegreenviper.com/miscellany/leaving-a-legacy/
[1] https://littlegreenviper.com/miscellany/concrete-galoshes/
[2] https://littlegreenviper.com/miscellany/testing-harness-vs-u...
I'm quite senior, and I don't think this is true.
Refactoring "as you go" is a no-go for a few reasons: (1) Roadmap (read: business) goals need to be reached and unless technical debt is a part of roadmap discussions -- it's usually not, because business people don't "get" code -- then it will naturally be of lower priority. (2) On a technical level, technical debt changes/re-arch should not live in feature branches. On a deeper technical level, technical debt pull requests and reviews will also usually incur a higher level of cognitive load (and thus will take longer to merge). Conflating technical debt and new features is a Bad Idea™. (3) This final point is a bit "20-20 hindsight," but with proper roadmapping and architecting (and unless you're building an MVP or proof of concept at your two-person startup), technical debt should basically not exist. Technical leaders should always push for high-quality code (even if product timelines are stretched) and technical debt is, generally speaking, a byproduct of bad technical leadership.