See also Chesterton’s fence. You don’t want to change something until you understand why it’s there in the first place.

https://abovethelaw.com/2014/01/the-fallacy-of-chestertons-f...

This reminds me of the Joel Spolsky's post on why you should never re-write your code from scratch [1].

The reasoning goes most 'ugliness' comes from bug-fixes that people encountered along the way, and by re-writing that 'two page function' you lose all that accumulated knowledge. In short, the hacks that make us want to rewrite code are there for a reason

[1] https://www.joelonsoftware.com/2000/04/06/things-you-should-...

That article by Joel Spolsky was one of the few I can't fully agree with. The greatest successes of my career have often been in creating "next generation" implementations of software using newer or better technology and designs. Those re-implementations have resulted in very significant performance and productivity gains. While rewriting things from scratch can definitely be a naive impulse, there are times when it's the right way to improve a product, and if you fail to do so, your competitors will. The trend towards microservices tacitly recognizes this, I think; one of the tenets being it should take a small team a couple of weeks to rewrite any component of the system from scratch. When software was written poorly to begin with and has grown unmanageable by layering hack upon hack, it can pay to take a step back, look at the actual requirements, and consider if there's a better way.

I wholeheartedly agree with what you said. There is a time and a place for leaving legacy code alone. I also know from first hand experience some things are just too far down the hacky rabbit hole and you just gotta cut of the dead stuff and start over sometimes. But on the other hand, the only times I've done that in my career the code was written in a very anti-pattern style pretty much against all language guidelines and had hidden side effects everywhere. It was a mess to figure out how some features worked so I just rewrote some features in 1/10th of the time it took the original maintainer to piece in similar bug fixes in the legacy code.

I have done my fair share of rewrites as well. The hardest ones are where you need the new system to be backwards compatible.

The piece focuses on ground-up rewrites of commercial products. If your project doesn't apply, then neither does his advice.

More likely the hacks are there because requirements changed and the software wasn't initially built to be flexible enough to support change.

A rewrite will solve it in the short term - until requirements change again.

However, I would much rather apply the new 'hacks' onto the rewritten 10 line function than figure out the original 200 line behemoth.

Each hack individually, yes, is likely to be there because of requirements changing.

But collectively, it's likely that one of those hacks is there to account for some edge case that's not intuitive and will be missed on the rewrite.

In my experience, it's extremely difficult to know which particular lines of code are obsolete. This becomes even more difficult as a codebase ages and is worked on by more contributors. Add on another exponent for every business stakeholder that has a hand in defining the business rules—especially if you have requirements coming from multiple sources that may not be aware of each other.

This is why we have the strangler pattern, right? You wrap the legacy functionality with a set of tests that define the functional requirements of the code, then your new refactored solution needs to keep those tests passing. (And then once you've done that, make sure that your "hacks and bugfixes" get test cases so you can make sure the next refactor accounts for them!)

Which is why comprehensive testing is important. If we have test cases that accurately capture our requirements, then we can refactor and know when we've err'd from the path. If we don't have test cases, then we have to make best guesses (and we know how that one turns out).

Accuracy is key, here. I came into my current job with some pretty awful tests in place that “passed” and showed good coverage but did absolutely nothing for actually hitting the necessary cases...

I have some code I inherited that I have rewritten it twice now and the second time is when it really became stable and also easier to work with.

Or you rewrite it to be more flexible and seriously think about what if's eg what if sales tax was changed in the middle of a month/billing period.

This is a real example at one job in the UK on budget day I used to listen the budget speech live in case it had any impact on the system I worked on.

I worked with one guy once where he wanted the "ultimate flexibility" and every data model object basically became a database table with the columns 'key' and 'value'. Trying to think of everything ahead of time has its own problems...

Which is how map/reduce systems work at the core - and how we implemented it back in the 1980's.

Each record type and field in that record had its own entry in a MIDAS table so you could process say TELEX records differently to Email, Data etc.

Actually, what that piece underpins is not that you shouldn't do rewrites, but that you shouldn't do that with the expense of the developer resources of your legacy application, you shouldn't put your old product "on hold" for the period of the rewrite, nor should you pretend that you are modifying the old product, but that rewrite effectively always results in a new product, and you shouldn't except that you can hotswap the new product with the old one.

The age old maxim "sell old products to old customers, and new products to new customers" works pretty well here as well.

How to rewrite the product if all your experts are tied up in the old product is a different question entirely.

For example of industry that seems to be in perpetual rewrite is the gaming industry. But they don't rewrite the old games, the develop new products using partly the existing concepts and software modules.

This product based view also tells you something: when in business, you should never do a rewrite out of purely technical aesthetic reasons, but because of concrete measurable business goals.

Rewriting the code from scratch does not necessarily mean clean-room reimplementation! It doesn't even mean you can't include parts of the original in the new code.

Rewriting from scratch means rethinking the design, or basic structure, of the code while keeping all its functionality. It's not about getting rid of hacks, it's about lining them up neatly in one place instead of having them all mixed, twisted, nested and threaded through the code everywhere.

Well, at least if you know what you're doing, instead of just randomly deciding on a rewrite because it's more interesting that way or because you can't be bothered to read and understand the original. Then, I agree completely with Joel, it's most of the time a grave mistake which made countless companies and developers fail.

> at least if you know what you're doing

Isn't this exactly the problem? I know "what I'm doing", but don't have the added knowledge of several engineeers through several years, and rewrite it's not necessarily a 1:1 transcription to a new language or structure.

If something is bothering the team daily, than yes, a rewrite might be needed, which is different, and hardly something I would exclusively by myself.

Exactly this. It’s usually much better to refactor/rewrite small pieces at a time where you can rather than taking a scorched earth approach on the entire codebase, and usually that involves including some of the “hacks” until you can get a better grasp on what the right design is.

My take on this is that it won’t help anyway. If there are problems in the code that have nothing to do with Chesterton, the fact is that the team wrote this code and asking for a do-over is magical thinking.

If you want to fix it, fix it. Find the other people who are fixing it and collaborate.

Advice I should follow myself: If they fight you on that, get out. They like their ball of mud, and they will turn new code into mud too. If you ever succeed, it will take you until it’s time to find a new job just to get things to tolerable. That’s an incredibly foolish investment in being right.

There are more people in the world that can be taught than there are teachers. Don’t waste your efforts on bad pupils. It just reduces our collective intelligence.

I think using "never" should be considered an exaggeration or embellishment, or else clearly disclaim that rewriting code given the existing needs of the business (including the existing code base) is different from writing brand-new code "from scratch".

Here's why: I saw a comment in some thread about machine learning yesterday that I think is a good analogy in some cases for what you describe. The comment essentially said that a "never re-write" policy leads to "overfitting" the code to the problems of the past and therefore less flexibility in implementing both fixes for issues and new feature development.

If it's well tested a straight re-write is the right call. It's when there isn't good test coverage where things ossify.

In theory. In practice, the code will often have corner case behaviours that eludes any of the tests, yet are still important.

As an aside, as I understand it, refactoring was popularised as a response basically to this conundrum - it's a technique for reorganising code without changing its functionality. So you can gradually "rewrite" your two page function without losing the accumulated knowledge.

Unless we really stretch the word "refactor", my "in practice" is different than yours. I've done plenty of re-writes that fixed more unknown bugs than re-opened previously solved ones.

I agree with other responses to this comment. Ugliness might have had a reason, ok. It's important to understand that. The question is whether it has a reason to stay. And here we find the usual balance between the effort it takes to maintain ugly code and the effort it takes writing new one. On another point... I understand calling that "accumulated knowledge". But that's knowledge accumulated very badly. You want to accumulate knowledge? Write tests when needed, comment them when needed, add comments when needed in the comment. That's accumulated knowedge too, and you are actually making it way easier to transfer, which is definitely one of the most important parts. If you were already doing things right, that's another story, but in general, if you look at code and your brain goes "what the..? no, please, why did you... T_T", there's probably a need for a rewrite.

EDIT: I think it's also important to mention the value in rewrites, specially on your own code. Spend a lot of time thinking how to make your code simpler, easier to follow, cleaner. That will help you write much better code in the future. A lot of people would benefit from this if they tried. Writing code that works is simple, writing good code is very hard. And rewriting code teaches a lot of the second. When you are learning, rewrite a lot. When you think you know what you are doing but someone suggests to rewrite (and you know they are good programmers too), at least listen.

> most 'ugliness' comes from bug-fixes that people encountered along the way

That's reality, sure, but there's no reason not to document it properly. Inadequate documentation seems to be assumed implicitly here.

If a (competent) reader is thinking Why on Earth does this code do that?, it means you've written unmaintainable code with too few comments around the necessary-but-ugly workarounds that you've used.

If you have properly commented everything, the reader won't mistake the necessary complexity of the code for a giant-ball-of-mud.

People have been measuring the cost of refactoring versus rewrite from scratch, but the results are, as so often in the industry of software development mostly ignored. Anyway, there (obviously) is a tilting point.

But what are the results?

if more than 25% of a component is to be revised, it's more efficient and effective to rewrite it from scratch. (It's fact 19 in 55 Facts and Fallacies of Software Engineering) which references several sources of it.

Mostly you shouldn't rewrite your code from scratch :-) - some times you need to like when we in the UK had to rewrite a billing system - which was a POS we had inherited from MCI.

Chesterton's fence has always bothered me a little because it presumes there will be a sensible explanation for the fence, and that the reason will reveal itself to us if we have the patience and guile to see it.

Knowing what I know about how some organizations work, I don't think this is necessarily a safe assumption.

I understood it differently. As in: there is always a reason for the fence but you never know whether it is justified or not without understanding it. Don’t tear down the fence until you understand why it was put there for the wrong reasons.

True, but what I think s_kilk was getting at is that sometimes (especially in software) a bad developer will put a "fence" somewhere without a reason. And if that person already left the company, maybe you can never find out why.

That's why it's important to start with some testcases of required behavior, and then refactor only code that is supported by tests. Because sometimes the fence is there for a reason, but often it's not.

Even bad developers have reasons, you may not be able to divine those reasons but they may be legitimate. The would-be refactorer would do well to consider carefully what the reason might have been, even if a git-blame reveals a less-than-excellent contributor.

As an programmer I always try to capture these reasons in my commit messages, after a decade of doing this I’ve gotten a lot of thank you notes years after leaving a project, and I’ve even reminded myself details about my own thought process that I had otherwise would have long forgotten.

One thing I’ve learned by having long tenures is that context change over years is much more likely the reason for nonsensical code than developer stupidity.

> some testcases of required behavior, and then refactor only code that is supported by tests

The difficulty is making sure that you have all the requirements and aren't missing any, and once a system gets big enough, that gets really difficult.

Chesterton’s fence is an idea to help you counteract the exact notion you are expressing here. You must prove that the reason for its existence is no longer sensible (or, perhaps, never was sensible) before removing it. There is no probability distribution associated with the proverb, except that there is a nonzero chance of sensible explanation.

No, the point of Chesterton's fence isn't presuming there is a reason:

> The more modern type of reformer goes gaily up to it and says, "I don't see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: "If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it."

The point is that there might be a reason, and before careful consideration we won't understand what that might be.

You can't expect it to just come to you, you have to ask. Unfortunately many organisations are bad at institutional memory, so reasons get lost. But fundamentally this requires a "why?" conversation, it can't be done by purely technical means.

If you're very confident that your test suite covers all use cases, then you can just change it and see what breaks.

A line I just used yesterday in a company presentation about feedback loops "if you don't understand a system's key feedback loops you don't truly understand the system"

We've got a new member on our team (from a different team within the company) who within his first week is already trying to shake things up with "fixes"

Its driving me a little crazy. He is more senior than us, but doesn't have any of the context of why we do things the way we do

He'll explain things to us we already know, and propose solutions to us we tried months ago that didn't/don't work

After getting a little frustrated, I pulled him aside at the end of the workday and sat him down to give him some context on a lot of things on our team, but he's determined to "shake things up" and believes he's "right" so "why should it matter"?

The one good thing about this is it's taught me just how patient and restrained I can be, which is a lot more than I thought, but please, please tell me that this eventually changes?

> The one good thing about this is it's taught me just how patient and restrained I can be, which is a lot more than I thought, but please, please tell me that this eventually changes?

It might not, but that's not necessarily a bad thing.

Personally speaking, I've been on both sides of this conversation. A number of years ago, I joined a company and I tried to shake things up with fixes. My initial ideas lacked the context, I didn't understand the constraints, and I certainly trivialized some important aspects of what was going on. Some people were patient with me and focused on explaining the unique context and constraints. Fortunately for me, I took the time to listen to them and I was in a much better position to offer up fixes.

Some of my fixes were good and well received, and others not so much. Typically, I found my ability to create positive change was strongly correlated to me taking the time to ask people questions, heeding their warnings, leveraging their unique insights and giving their contrasting opinions credit.

When I hired a technical lead for my team, the shoe was on the other foot and when he joined. He offered a lot of very good ideas to shake things up and fix things. When he asked for permission to do these things, I made it a point to never say no, but to take the time to explain the context and constraints. His first few months weren't tremendously productive because we subjected him to the pain of the thing he was going to eventually automate. Eventually, I was given the opportunity to let him loose and give him the freedom to do what he thought was important. I'm glad we did, because he ultimately created some really good tools. More importantly, he was able to prove me wrong about some of the problems I thought were unique to us.

I guess what I learned from it was the importance of directing people's energy. As you already know, it requires the patience to continuously engage in conversations to make sure that self-drive isn't wasted. These days, I'm a lot more appreciate of someone joining the team has a strong locus of control.

If you really want him to become part of the team, try distributing the task by encouraging other team members to help him. He'll be a lot more receptive to listening to your unique constraints if it's coming from people who are trying to help him.

Guys, should we tell him?

Oh, no

What am I missing?

It won't get better, he's more senior, he'll probably get promoted before you guys (and he has a head start) so he'll be your bosses boss and mandate all this stuff.

You'll leave because surely everywhere can't be this dysfunctional only to find that everywhere is then you'll reach a grudging acceptance (people will call you cynical) and you will live for your hobbies.

40 years from now a vein blows out and you shuffle of the mortal coil.

Alternatively it'll get better, he'll realise that his ideas while well intentioned are upsetting the team and he'll tone it down a bit.

Flip a coin. (I wish I was joking about the binary options but well...it'll be one of those two).

Everything changes. But it's likely to be the whippersnapper moving on (again), while the rest you you gets to mend all the semi-implemented best practices to resemble some sort of coherent whole again. Or not. Nah, you're probably ok.

On the plus side, his ideas should give you a moment to reflect on whether or not the way you do things is still the right way. Sometimes you need fresh ideas and approaches. Unfortunately, it's harder to hear when forced like this.

Hmm, well tell us the ideas and we'll tell you if he's right or not.

I was asked in an interview "What is the first thing you will change?" (I was interviewing for a Manager position, but I think this applies no matter the position you are being interviewed for.)

My response was - "I don't know. I need to spend time getting to know the team, understand our product, and better understand priorities."

I try to use this question in interviews with senior people, be they Developers or Managers. It can expose someone who will quickly blow your team up should they join. I like working with people who have great passion and will stand up for their ideas, but understanding the problem first is alway wise!

Getting to know people, products, and customers, (not that customers aren't people) with listening and questions makes everyone better.

> "What is the first thing you will change?"

My response: "My mind" ... As i seek to _first_ understand and then to act...

10 Points

This honestly took me a while to learn - it's so easy to jump in and be a smartass (and I've been guilty of this before) about how bad things are, but a) suggesting small, incremental changes that demonstrate a nuanced understanding of where things went wrong and b) volunteering to work on making things better goes a long way. Most of the team will already often know and agree on what is bad - but just won't be empowered to fix it somehow.

> This honestly took me a while to learn

Yeah same here. I think I pissed off my manager pretty quickly at my current job by trying to change how they did certain things before getting an understanding of why they did them that way, but I realized my mistake and went into more of an observer-mindset for awhile.

Over time I built up a reputation with reducing friction in smaller ways on specific features I worked on, and then applied that same thinking to larger scale issues I saw. They were much more receptive once they knew I could make valuable contributions, and now I'm one of the main people establishing patterns and architecture decisions at my company.

Learning to establish myself for awhile before making process changes was a great learning experience that I plan to take with me to all future roles.

Your comment resonates with me.

I work hard to improve things; most of it was worth it, some of it didn't went well but it was a constant fight with my boss. If I failed, he would bring it back for months.

What worked for me was to step back and get into super pasive mode. Give all the responsability to my manager. I didn't question any of his design decisions, gave support to all ideas even if seem plain wrong unless he wanted honest feedback (usually, he didn't because I would change the original idea). Never say or even insinuate "I told you this was not gonna work"; I just acted surprised and asking to him "ok, what do we do now?". It just took a few months for my manager to start reliying on me, more and more. Now I have even more space for improving things, with the full support of my manager because "he wants me to do it" not "because I push my ideas to him".

At the end of the day, it's not dumb if it works.

Agreed. I have a theory that the reason contractors want to change everything when they arrive is so that they have infinite work.

I also always see contractors wanting to start self-initiated projects from scratch rather than working in the codebase, which is of course easier to produce a visible result in.

As a person working in this scenario I find the biggest issue coming into work on an established code-base is the lack of documentation and a lack of interest from the company's own people to divulge any information about the code base. This unfortunately leads more often than not to the situation you describe, re-written code or projects perhaps unnecessarily separated.

Very true as well, and if that were plainly stated the problem could be addressed. The issue is that some of these people are smooth talkers able to convince managers that these big shifts are necessary right away...and inevitably end up abandoning big projects they push for.

Others are able to poke around the codebase and ask questions till they can contribute in a meaningful way, but these are rare in my experience.

> Very true as well, and if that were plainly stated the problem could be addressed.

That's not, at all, how this works, especially for newcomers.

If the new programmer on your team doesn't want to work on a larger code base, it's your responsibility to notice this early, initiate a talk with them about the reasons and actively try to fix the problems they have.

Reading code is hard enough, but reading it under pressure of being the new guy makes it even worse. Don't expect many people to be able to cope with this without serious effort on your part. If you can't be bothered to effectively support them in understanding your code, they are not likely to care about that code sufficiently to productively work on it.

When you come in on a contract you need to fix or add something specific. Keeping the existing codebase is usually last on the list of priorities.

When you're brought in to add something at a critical time but instead push for bigger changes because of inability to work with an existing codebase that means they need to find a better programmer for the time being to do the role you were hired for.

It's rare that existing employees don't realize a codebase is crap, and starting over they could do better...that's not why contractors are hired, to state the obvious.

> Process and code fixes are _much_ easier once you have good rapport with the team.

And one doesn't get that by walking into a place expecting to make sweeping changes.

What's worse is job hoppers were asked so this means people are going into a shop, making these changes, and leaving shortly after. The hell?

More realistically they’re going into a shop proposing changes immediately, getting slapped down, becoming disillusioned, leaving, and then trying again somewhere else. Even when people do make changes like this, it’s unlikely they stand the test of time, I’ve definitely seen people’s “pet changes” get rolled back within microseconds of them exiting.

You're absolutely right. "job hoppers" are often the very motivated people that are worth their weight in gold, but don't find a company that is on their level.

However as a manager you should probably not institute sweeping changes a "newbie" (he could actually be more of an expert than your own team, but you don't know this yet) suggests, but finding out why the newbie suggests them should be in your best interest. Often fresh eyes are the only ones that can see what is wrong. Dan Luu put this really well: https://danluu.com/wat/.

A lot of catastrophe's could have been avoided (especially stuff that's morally questionable) if people listened more to the newbie who still isn't conforming enough to your team that he still sees what's wrong.

So my advice (to the managers): listen to the suggestions, and put your managerial clout and preexisting rapport with your team (which the newbie lacks) to implement the good ones, if there are good ones.

If they aren't you can still explain why.

+1 for this.

Unless you've been hired specifically to help them change their dev practices, I'd go along with what they have until you've got some understanding and reputational-clout to start suggesting such huge changes to how everyone works.

Rocking up and on day one start asking people to change their development practices (particularly around branching policy which is something very contentious and/or strict everywhere I've been) because "It makes my work easier" is a sure-fire way to get people's backs up and dislike you. And where do you stop? First it "It makes my work easier" to change the dev practices, but why not start demanding that everything is rewritten in Go/Vue.js/Haskell? It'll make your work easier. How about we change the product from a Desktop App to a web app? It'll make your work easier. Why dont we just sub-contract the whole thing out to off-shore teams? It'll make your work easier. There might be a case for any of those, but I'd personally wait (and I would prefer any hires I bring in also wait!) until the right time before advocating for massive sweeping changes right away.

tl;dr - Arrogance & know-it-alls can be disruptive (in a bad way). It takes a village and flexibility is key to working with any team.

The fact that process is off in the first place indicates a deeper problem.

If the process is 'off' around branching policy, there may actually not be a problem. It's certainly possible the new dev prefers a different branching policy simply because that fits with what they are used to. There may be reasons, and good ones, for what may at first appear like poor git practice. Or it may be a problem, but a new dev should at least give it enough time to determine there is no reason things are this way before advocating change.

In which case the correct answer is to sit tight for a bit until you understand the deeper problem. And then you fix the deeper problem, and then you fix the process.

This is one I learned the hard way. I made a number of much-needed fixes at a job where they'd never really had a dedicated technical person before. People freaked out. I hadn't communicated what I was doing and why it was necessary, and it took a while to earn goodwill after that. Lesson learned: don't implement a solution until you've convinced everyone they have a problem and this will fix it.

I was going to say "Happy Hour" but this is really what I meant. I focus first on understanding how the team works, their motivations, their past pain points, and honestly become one of them before I suggest changing what is then _our_ process.

I wouldn't, on the first day, try to take up more of my coworkers time. I would wait until I learned the norms about after-work outings and what sort of obligations they have outside of work.

I whole-heartedly agree. I've worked with too many Senior Engineers who immediately want to change processes in their first two weeks of starting at a new company. It only alienates the other developers.

Yeah, I have to say I'm a bit surprised to see people so eager to change things that are working, so fast.

Sometimes those things that are working imposed a hefty load of technical debt, which newcomers are tasked to pay up with compound interest due to the unfamiliarity with the codebase. Therefore rewriting some components may actually pay off in productivity in the short run.

'Everything got the way it is one logical step at a time' - paraphrase of some of Gerald Weinberg's advice in 'The Secrets of Consulting'

Adam Grant talks about this in "Originals".