News for hackers. Stuff that matters.
Re:We invested 10% to pay back tech debt; Here's what happened by kace91 on Sunday January 15, 2023 @07:04PM (#34395079)
I’ve seen “x% time for tech debt“ rules in several companies and sadly it didn’t work too well.

Since the problem was the culture of continually pushing half baked features in the first place, the rule was quickly corrupted: people would design a good system, throw anything that’s not required for a POC into the tech debt backlog and deliver a barely functioning version.

“This is a technical debt task” was used to prevent everything that wasnt new Features taking time of the other 90% of the sprint.

Basically, if you assign a block of time to quality, you risk people taking that as an excuse to not focus on quality outside that block.

Re:We invested 10% to pay back tech debt; Here's what happened by theptip on Sunday January 15, 2023 @07:38PM (#34395323)
I think the key observation here is if the team doesn’t care about quality, you can’t create it out of thin air.

If the team does care about quality (as described in the OP), then something like a tech debt budget/carveout can be a good management/scheduling strategy to buy the team breathing room from the rest of the org.

I have used this strategy successfully in the past. For some reason it’s often easier to “spend 10% of time on tech debt” than “spend 10% of time polishing your code to avoid tech debt in the first place”. I don’t even think the latter is the correct way to build software, as you seldom know ahead of time what will justify continuous polish and refinement.

The advantage of discretionary tech debt fix time is it lets you gradually refine rough edges as they become pain points, with a low-friction bottom-up process (ie the developers that see the rough edge are empowered to fix it, rather than requiring PM/scheduling overhead for everything).

Re:We invested 10% to pay back tech debt; Here's what happened by neilv on Sunday January 15, 2023 @07:17PM (#34395170)
I've also seen this sprint-task-fake-completed later cleanup misattributed to "refactoring", which I've started calling "fudge-factoring".

IMHO, at the point you have engineers misrepresenting that something is done, even if you're not in an application domain in which engineers go to jail for that, you take a step back and fix your engineering/product culture.

That might include nuking your current methodology from orbit, just to reinforce that organization needs to learn a new way of thinking. (Not just "don't do that" on one symptom, when whatever culture weakness that caused the fudging is still in play.)

Re:We invested 10% to pay back tech debt; Here's what happened by josephg on Sunday January 15, 2023 @07:42PM (#34395349)
Yep. Or in other words, you (usually) can’t solve cultural problems with technical solutions. If your team culture doesn’t value long term thinking, it’ll take more than a schedule change to introduce that mentality.

In the story in this post, one important thing that happened was that the team (with management) came together to acknowledge the tech debt problem, acknowledge they want to solve it and talk through potential solutions. This moves the problem from something individuals care about to something the team as a whole acknowledges it care about (a shift from individual knowledge to common knowledge). Once everyone agrees this is a problem, individual engineers will know they’re acting against the will of the tribe by writing lazy pull requests. And being reprimanded by their coworkers for doing so will have much more weight.

Re:We invested 10% to pay back tech debt; Here's what happened by eru on Sunday January 15, 2023 @09:05PM (#34395953)
> Or in other words, you (usually) can’t solve cultural problems with technical solutions.

Yes, though technical solutions go a long way towards making cultural solutions work and stick.

Eg running your tests automatically when someone makes a Pull Request is much more robust, even in a cultural sense, than asking developers to manually run the tests.

Technical solutions can help make the culturally favoured approach also be the path-of-least resistance.

Of course, you still have to write the tests. (Though even there, you can make your technical tools check for eg test coverage. That can be gamed, but that's more effort than when you can just ignore it; and even gamed coverage is better than no coverage.)

Re:We invested 10% to pay back tech debt; Here's what happened by hgsgm on Sunday January 15, 2023 @09:19PM (#34396039)
"Locks on doors keep honest people honest". Same goes for rules in an organization. It gives people an excuse to do the right thing.
Re:We invested 10% to pay back tech debt; Here's what happened by josephg on Sunday January 15, 2023 @09:54PM (#34396274)
True; but you're going to get much better testing if your team collectively cares about it than if it feels externally mandated by that one annoying guy who set up your commit rules.

Technical solutions are the best solutions when they work. But being technologists will bias us towards looking for technological ways to solve problems. Its not always the best approach. A few hours chatting over lunch is usually a much more effective way to change how your team works.

Re:We invested 10% to pay back tech debt; Here's what happened by wpietri on Sunday January 15, 2023 @11:39PM (#34396866)
> True; but you're going to get much better testing if your team collectively cares about it than if it feels externally mandated by that one annoying guy who set up your commit rules.

Some of the worst tests I've ever seen were in places where managers measured test coverage. Reams and reams of absolutely useless tests that may not have checked anything that mattered, but that did bump the coverage metric up.

Re:We invested 10% to pay back tech debt; Here's what happened by quanticle on Monday January 16, 2023 @01:50AM (#34397650)

    Reams and reams of absolutely useless tests that may not have
    checked anything that mattered, but that did bump the coverage
    metric up.
I would go farther than that, and call those tests worse than useless. Tests that are written to appease some kind of automated coverage metric god tend to be tests that test internal implementations rather than API contracts. So you end up with a codebase where a single line change to production code requires something like 150 lines of test code changes, all because the internal representation of some data was changed in some kind of trivial way.

It's possible to avoid this failure mode, but the teams that are smart enough to avoid this failure mode are also generally smart enough to avoid having unreasonable coverage goals.

Re:We invested 10% to pay back tech debt; Here's what happened by F-W-M on Monday January 16, 2023 @06:52AM (#34399715)
I once found

``` if (var) assert(var) else assert(!var) ```

in the outsourced test suite for a level A air plane software.

Re:We invested 10% to pay back tech debt; Here's what happened by eru on Monday January 16, 2023 @01:05AM (#34397368)
Agreed. I am suggesting that the technical environment can make certain cultures mores easier or harder to do.

Cultural change is hard enough, so getting as many of the technical nudges as you can, is good.

Re:We invested 10% to pay back tech debt; Here's what happened by Retric on Sunday January 15, 2023 @08:23PM (#34395666)
I think it’s less an engineering culture issue than a management one. Do you reward people based on number of features, meeting rapid deadlines, quality, or something else like customer satisfaction?

People respond to an organizations actual priorities not whatever platitudes people toss out. If you want quality you need to sacrifice something even if long term results improve in the short term quality takes time.

Re:We invested 10% to pay back tech debt; Here's what happened by wpietri on Sunday January 15, 2023 @11:37PM (#34396852)
Exactly. I recall one place where an exec pushed for something to be completed in a too-short timeframe. It launched on the schedule he announced to his higher-ups, so he was promoted and moved elsewhere. Woe be to the exec who inherited the project, though, as the code was basically a mound of turds wrapped in Christmas paper. And the team, of course, was fractured and burnt out. The cherry on top was watching the ladder-climber throw a lunch to thank the team; their reward for hundreds of hours of grueling overtime was lukewarm Chinese food dished out of large aluminum bins in a conference room. Plus $50 gift cards!

This was a long time ago, but I still think about it as a moment when I learned what POSIWID really meant.

Re:We invested 10% to pay back tech debt; Here's what happened by tjrDL6MjB2Zwwa on Monday January 16, 2023 @06:34PM (#34407040)
heh, what is POSIWID?
Re:We invested 10% to pay back tech debt; Here's what happened by wpietri on Monday January 16, 2023 @09:28PM (#34408458)
Re:We invested 10% to pay back tech debt; Here's what happened by hiddencost on Sunday January 15, 2023 @08:47PM (#34395825)
I've watched a really elite group of engineers fall apart slowly because one leader decided to start committing systematic fraud and coaching his delegates to commit fraud.

It's well done, because it's almost impossible to prove.

Those of us who still care about system health and helping users just wait two years after they launch something, ask leadership for permission to delete it, and keep trying to build abstractions that reduce the harm misconduct has on our parts of the system.

The problem is they keep getting promoted because no one notices that almost nothing they've built is still in production.

A few of us have tried raising the issue to leadership but no one in leadership wants to know. It's a bit spooky. My current theory is that no one has the power to deal with it anymore, and at higher levels, launching garbage features that get deleted is actually desirable.

Theory: senior leadership keeps saying "drop what you're doing and build X". This group does that, but in a way that's half baked enough that we can kill it later. Middle management benefits because senior leadership is happy, and deleting it is fine because leadership forgot about it.

I wish I knew a way out.

Re:We invested 10% to pay back tech debt; Here's what happened by mansoon on Sunday January 15, 2023 @08:54PM (#34395885)
Leetcode
Re:We invested 10% to pay back tech debt; Here's what happened by neilv on Sunday January 15, 2023 @09:05PM (#34395955)
Leetcode, meaning here to switch jobs?

A little ironic, because Leetcode interviews arguably send the message that the org is about nonsense hoop-jumping and politics.

Re:We invested 10% to pay back tech debt; Here's what happened by zeroonetwothree on Sunday January 15, 2023 @10:39PM (#34396521)
Leetcode seems like the opposite of politics at least. Hoop jumping sure.
Re:We invested 10% to pay back tech debt; Here's what happened by sidlls on Sunday January 15, 2023 @11:05PM (#34396663)
It's a signal of politics, too: at companies where the people who view that kind of hoop jumping as an important signal got their way, it means they won that political battle.
Re:We invested 10% to pay back tech debt; Here's what happened by sebzim4500 on Monday January 16, 2023 @08:36AM (#34400464)
I don't get what you mean. Couldn't you equally claim that in companies that avoid leetcode interviews the anti-leetcode people won that political battle. I don't see why one option is more political than the other.
Re:We invested 10% to pay back tech debt; Here's what happened by sidlls on Monday January 16, 2023 @01:00PM (#34403495)
You're right, of course: the politics works both ways here.
Re:We invested 10% to pay back tech debt; Here's what happened by gymbeaux on Monday January 16, 2023 @03:46PM (#34405274)
I’m sure the topic of leetcode has been beaten to death on a forum such as this, but serious question: with ChatGPT, why WOULDNT I cheat, assuming the leetcode is the unmonitored/unproctored variety (most are this way)? Like seriously. We know the leetcode skill isn’t applicable to the actual duties and responsibilities of the job itself, so why not cheat, especially when ChatGPT makes it so easy?
Re:We invested 10% to pay back tech debt; Here's what happened by raincom on Sunday January 15, 2023 @10:21PM (#34396424)
If organizational goals are centered about building X with half-baked features and then kill it after the perf review cycle, it is rational for everyone to focus on that. Stop wasting time on fixing tech debt. There is always a main product that pays all salaries and bonuses. As organization gets larger, fiefdoms get built by hiring more people. Just understand what’s going on and use that “knowledge” to advance careers and to make more money.
Re:We invested 10% to pay back tech debt; Here's what happened by karmakaze on Sunday January 15, 2023 @10:32PM (#34396485)
I have a knee-jerk reaction when most people say "refactoring". In many cases they're talking about a different, arbitrary way to deduplicate an implementation, usually in a mindless/mechanical way. If you can't name the 'factor' that you're separating and that factor doesn't have business or conceptual cohesiveness, then you're just pushing bits around for someone else to want to do it differently. So now when someone says it, I always ask what do you propose to factor and why?
Re:We invested 10% to pay back tech debt; Here's what happened by rileymat2 on Monday January 16, 2023 @12:28AM (#34397153)
> usually in a mindless/mechanical way

Loaded language not withstanding one of the premier books is all about “mindless/mechanical” changes to structure without changing behavior. It is about step by step safe microchanges.

This might be the biggest difference between refactoring and rewriting.

But you are right it is pushing bits around with no behavioral change. That’s the point.

The open question is if this newly refactored software is more maintainable, more understandable, easier to change in the dimensions it will mutate in the future. Often the answer is “no” and you are right to be frustrated.

Re:We invested 10% to pay back tech debt; Here's what happened by karmakaze on Monday January 16, 2023 @12:50AM (#34397284)
The execution (how) can be 'mechanical' but the goals and choices (why) should be thoughtful. If it's simply deduplication there is no thoughtful why necessary, although in the worst cases abstract machinery is created for the purpose.

You're right about my misconstruing 'no behaviorial change' as that is always a requirement. There should however be a change in where the seams of the implementation lie which better suits the direction of expected changes.

Re:We invested 10% to pay back tech debt; Here's what happened by escapedmoose on Sunday January 15, 2023 @10:36PM (#34396509)
When the team has gone that far off the edge, imo it’s good when the wheels start falling off, when tickets start piling up, when the dev team very visibly has egg on its face. Sometimes it takes until the team is embarrassed before management embraces the option to go nuclear on the dev methodology. But I don’t know how to ensure that it’s the poisonous methodology that’s targeted and not individual people/scapegoats.
Re:We invested 10% to pay back tech debt; Here's what happened by switch007 on Monday January 16, 2023 @04:46AM (#34398766)
> I've also seen this sprint-task-fake-completed later cleanup misattributed to "refactoring", which I've started calling "fudge-factoring".

Having just spent a week clearing up and properly implementing my colleagues' 'fake completed' sprint tasks, I can resonate deeply. I'm making it clear to my boss that it wasn't 'refactoring', I was just implementing the actual requirements properly.

I'm going to go over their next PR with a fine-tooth comb, no matter how long it takes

Re:We invested 10% to pay back tech debt; Here's what happened by anothernewdude on Sunday January 15, 2023 @07:56PM (#34395451)
I've seen some success with targeted reductions in tech debt. Limiting the scope, limiting what can be changed, and focusing on a goal to be achieved by reducing the tech debt. It also makes it a much easier sell to the client.
Re:We invested 10% to pay back tech debt; Here's what happened by NomDePlum on Monday January 16, 2023 @01:18AM (#34397461)
I agree with this. I also find it useful for the team once they have identified an area or specific aspect of what they perceive as technical debt to tackle to take the various ideas they have for tackling it and do a very rough value Vs effort exercise together.

Just having that conversation can reduce both perceived technical debt (people explain to other people why it works in that way and that's the right solution) but also team angst about the issue as they get to tell other people about it.

Whatever is the easiest, highest value things normally pop out and you can slowly work through them with better understanding.

It also means you don't start work on the hardest to solve peeve of a single individual just because they are the loudest voice.

Re:We invested 10% to pay back tech debt; Here's what happened by ryanbrunner on Sunday January 15, 2023 @07:57PM (#34395459)
I think an important aspect is separating "tech debt time" completely from your backlog or planning. Generally speaking, there's less stakeholders involved in reducing tech debt and less need for requirements gathering or prioritization - developers generally know where the worst problems are and can be relied upon to choose something to focus on next.

I'd avoid the temptation even to have "projects" during that time - make it purely about accomplishing refactors and tech debt removal that can be completed during that day, and specifically don't have plans or a backlog.

Re:We invested 10% to pay back tech debt; Here's what happened by bcrosby95 on Sunday January 15, 2023 @08:51PM (#34395860)
If you have engineers doing that, you have bigger problems on your hands.
Re:We invested 10% to pay back tech debt; Here's what happened by karmakaze on Sunday January 15, 2023 @10:26PM (#34396455)
Thanks for pointing this out--it would clearly be the case doing the thought experiment and applying human behavior around shipping timelines. I may be in a position to dedicate some effort to reducing tech debt but I don't want it to move the goalposts so it's ok to ship more of it.

Instead I want to pick out the worst culprits that bother most devs and in particular onboarding new ones and deal with those.

Re:We invested 10% to pay back tech debt; Here's what happened by quickthrower2 on Sunday January 15, 2023 @11:31PM (#34396821)
The other problem with “tech debt” time (and same with “devops”) is it becomes a bucket for all kinds of stuff that isn’t tech debt at all. For example upgrading framework versions, fixing bugs, and so on. Not necessarily wrong to do these instead but it is kind of the team fooling itself.

Instead I think you need the unicorn of someone who understands product and engineering and can decide at any given time whether to pay down tech debt or add a new feature.

Re:We invested 10% to pay back tech debt; Here's what happened by ebiester on Monday January 16, 2023 @02:40AM (#34397957)
This is why I prefer a balance of "Product Initiatives" and "Engineering Initiatives" and that number has to be bigger than 10% because of all that it encompasses.

The hard part is explaining to the business the return on investment on engineering initiatives because often times, product thinks of this as a zero sum game. They really care about the sum output of features, but it gets conflated with the number of hours placed in product development.

It is engineering management's task to break that mental model.

Re:We invested 10% to pay back tech debt; Here's what happened by mmcnl on Monday January 16, 2023 @05:00PM (#34406034)
I honestly think they're the same. If you think they're not then you simply haven't thought it through well enough, which means you need to spend more time on that.
Re:We invested 10% to pay back tech debt; Here's what happened by ebiester on Tuesday January 17, 2023 @01:16AM (#34409650)
There are some subtle differences, but framing is important.

Debt has connotations of digging yourself into a hole. If you are in a new project, you likely have all sorts of tools that you would like to build. You are not in debt to start - you have no code!

This gives latitude for thinking ahead and it also allows for including things like enablement and platform teams. Debt retirement, in this view, is a subset of technical initiatives. And some of that technical initiative budget should also go to product teams.

Re:We invested 10% to pay back tech debt; Here's what happened by mmcnl on Tuesday January 17, 2023 @03:39PM (#34418631)
That is true but framing it as technical budget means there is no business benefit. This is false. Long term maintainability and increased innovation speed are clearly business benefits that you hide behind ambiguous terms if you use wording such as technical debt. Just be specific. Technical debt isn't specific.
Re:We invested 10% to pay back tech debt; Here's what happened by jollofricepeas on Monday January 16, 2023 @12:30AM (#34397163)
I knows it’s controversial but…

Tech debt and house repairs are the same.

Plumbers, electricians, roofers, painters and the like will all encourage you to handle your repairs timely and proactively do maintenance.

But in the end, it’s the owner (shareholders) of the home that is best qualified to make maintenance decisions based on their needs, wants and resources.

We as developers are service professionals (contractors) and it’s not unless you also have equity that you will have a better sense of how to handle tech debt appropriately.

Bring on the downvotes :p

Re:We invested 10% to pay back tech debt; Here's what happened by vgel on Monday January 16, 2023 @01:16AM (#34397447)
The problem is a lack of trust, same as a contractor. If you go to a new mechanic and he says you need a $1,000 repair—do you? Or is he trying to sell you replacement blinker fluid?

A lot of managers feel the same way when engineers talk about tech debt, refactoring, etc. Maybe they've never read code, or aren't familiar with the specific codebase. Projects seem to be getting completed—what's the problem? Maybe velocity is going down, but are the engineers right about why? Or are they just being anal about something that's simply less than ideal?

A good contractor walks you through why the repair is needed, what your options are (cheap temporary fix, long-term repair, total replacement), the consequences of those choices, and how much each will cost.

Unfortunately in software we don't really know how to answer any of those questions. A lot of refactorings and rewrites just shuffle irreducible complexity around ("it makes so much more sense now!" says the developer who just spent a week studying the code and rearranging it to their personal preferences). Not to mention that we suck at giving estimates.

Re:We invested 10% to pay back tech debt; Here's what happened by sumanthvepa on Monday January 16, 2023 @02:09AM (#34397778)
One reason why your CEO needs to be technical. I run my own small software shop, and I can sit with my developers and actually see that the code needs refactoring. Indeed, it is often me (the CEO) that actually forces a refactoring sprint on the team. They usually want to move on to the next beautiful feature. But long term, our feature velocity and product quality suffers when the internals are bad.
Re:We invested 10% to pay back tech debt; Here's what happened by krageon on Monday January 16, 2023 @04:07AM (#34398540)
> Unfortunately in software we don't really know how to answer any of those questions.

Good senior engineers can absolutely answer these questions. Is this not what you've seen in Enterprise software dev? I'd say if you're a senior engineer in this context it's more than half of your job.

Re:We invested 10% to pay back tech debt; Here's what happened by mmcnl on Monday January 16, 2023 @04:58PM (#34406019)
> A good contractor walks you through why the repair is needed, what your options are (cheap temporary fix, long-term repair, total replacement), the consequences of those choices, and how much each will cost.

This 100%. Technical debt is placeholder term for engineers who want to do stuff but don't want to spend time explaining the people paying their salaries why they are doing it. Technical debt is meaningless. Be more specific. Present options. Be a professional?

Re:We invested 10% to pay back tech debt; Here's what happened by somethingreen on Monday January 16, 2023 @04:55AM (#34398834)
I disagree with the analogy. Need for repairs arises because of the natural decay. Tech debt is a consequence of the decisions made at implementation or even planning stage. It's more like forgetting to order a plumbing connector and using duct tape instead. Except in software there often isn't any forgetting involved and the actual connector is never ordered.
Re:We invested 10% to pay back tech debt; Here's what happened by towawy on Monday January 16, 2023 @07:16AM (#34399865)
A lot of tech debt is self-inflicted, but not all. Decay is induced by external forces and the same thing can happen in software, e.g. when underlying platforms change.

Mobile apps are pretty high maintenance because you often have to change something with new OS releases and want to roll out an update. You can abstract over the platform but that is in itself costly, so there is a tradeoff.

Re:We invested 10% to pay back tech debt; Here's what happened by mgkimsal on Monday January 16, 2023 @09:22AM (#34400903)
Bad analogy. Whether I 'own' the company (equity/etc), if I'm a full time employee, the maintenance/debt needs of the company still affect me. They possibly affect me more than someone who merely has 'equity' in the company (people may own shares in multiple companies). A 10% drop in the company's revenue might mean I lose my job.

If the painters/electricians also lived in the house, and it was their only/primary shelter, the 'owner' isn't the only one 'best qualified' to make maintenance decisions.

Re:We invested 10% to pay back tech debt; Here's what happened by jollofricepeas on Monday January 16, 2023 @11:01PM (#34408979)
“Might lose your job” isn’t the same as having equity or even having 50-100% of your salary based on revenue.

Traditional workers have the job market as backup.

The incentives do not align.

Re:We invested 10% to pay back tech debt; Here's what happened by mmcnl on Monday January 16, 2023 @04:54PM (#34405976)
I think separating tasks for features and for quality is wrong. They are not two different things. There is only purpose, and that is to serve the user's needs. Therefore I also think technical debt is the wrong way to phrase it, because it takes away the need to express the task in terms of what it means to the user. If a tree falls in the forest and no one is around to hear it, does it make a sound? If the user won't notice technical debt, then who cares?

You can refine technical debt by simply asking the question: what will the user notice if technical debt is not repaid? Use the answer to that question to phrase the task and suddenly it's not technical debt anymore, but actual work that the business will prioritize (or not).