You’ll enjoy the Big Ball of Mud paper[1].
Real world systems are prone to decay. You first of all start with a big ball of mud because you’re building a system before you know what you want. Then as parts of the system grow up, you improve the design. Then things change again and the beautiful abstraction breaks down.
Production software is always changing. That’s the beauty of it. Your job is to support this with a mix of domain modeling, good enough abstraction, and constructive destruction. Like a city that grows from a village.
[1] https://laputan.org/mud/mud.html
[2] my recap (but the paper is very approachable, if long) https://swizec.com/blog/big-ball-of-mud-the-worlds-most-popu...
https://s3.amazonaws.com/systemsandpapers/papers/bigballofmu...
I think anyone that thinks mudball is OK because business is messy has never seen true mudball code.
I've had to walk out of potential work because after looking at what they had I simply had to tell them I cannot help you, you need a team and probably at minimum a year to make any meaningful progress. That is what mudballs leads to. What this paper describes is competent work that is pushed too quickly for cleaning rough edges but has some sort of structure.
I've seen mudballs that required 6-12 months just to do discovery of all the pieces and parts. Hundreds of different version of things no central source control, different deployment techniques depending on the person that coded it even within the same project.
I’ve seen and created some pretty bad stuff. Point is not that it’s okay, but that that’s the job: managing, extending, and fixing the mess.
Yes a perfect codebase would be great, but the code is not perfect and there’s a job to do. You’re not gonna rebuild all of San Francisco just to upgrade the plumbing on one street.
Much of engineering is about building systems to keep the mess manageable, the errors contained, etc. And you have to do that while keeping the system running.
I've seen numerous places trying to hire someone to fix a 5-10 year mudball that has reached a point where progress is no longer possible without breaking something else which breaks something else and so on.
There is an endgame to the mudball and it does end in complete and total development stopping and systems that are constantly going offline and take weeks to get restarted. Most of the time the place will say: "Oh we've already had several consultants tell us the same thing" The same thing being the situation is hopeless and they are facing years of simply untangling the mess they made.
Usually the mudball is held together by a chain of increasingly shorter senior positions that keep jumping the sinking ship faster and faster. Finally they can no longer convince anyone sane to take on the ticking time bomb they have created and they turn to consultants.
Also my advice is often you should bring back person X that was at least familiar with the system at whatever salary they require. I am inevitably told that that person will literally not even take calls or emails from the company any more, every time. Thats how bad a real world mudball is.
Instead, at least one implementer needs to get hands dirty on what the application space really is. Very dirty. So dirty that they actually start to really know and care about what the users actually experience every day.
Or, more realistically for most companies, we insist on separate silos, "business logic" comes to mean "stupid stuff we don't really care about", and we screw around with if statements. (Or, whatever, we get hip to monads and screw around with those. That's way cooler.)
When the problem itself is technical or can be generalised then abstractions can eliminate the need for 1000s of if-statement developers but if the domain itself is messy and poorly specified then the only ways abstractions (and tooling) can help is to bake in flexibility, because contradiction might be a feature not a bug...
You're right that the business logic is gonna be messy, and that's because nobody really cares, and they can offload the responsibility to developers, or anyone punching it in.
On the other hand, separating "good code" and "bad code" can have horrible outcomes too.
One "solution" I saw in a fintech I worked at, was putting the logic in the hands of business people itself, in the form of a decision engine.
Basically it forced the business itself to maintain its own ball of mud. It was impossible to test, impossible to understand and even impossible simulate. Eventually software operators were hired, basically junior-level developers using a graphical interface for writing the code.
It was rewritten a couple times, always with the same outcome of everything getting messy after two or three years.
It would just keep adding what it called "heuristics", which were just if statements that tested for a specific condition that arose during the bug. I could write 10 tests for a specific type of bug, and it would happily fix all of them. When I add another one test with the same kind of bug it obviously fails, because the fix that Codex came up with was a bunch of if statements that matched the first 10 tests.
I am convinced this behaviour and the one you described are due to optimising for swe benchmarks that reward 1-shotting fixes without regard to quality. Writing code like this makes complete sense in that context.
Thank you for giving a perfect example of what I was describing.
The thing is, you actually can make the software work this way, you just have to add enough if-statements to handle all cases--or rather, enough cases that the manager is happy.
I'm firmly in the "DI||GTFO" camp, so I don't meant to advocate for the Factory pattern but saying that only abstractions that you like are ok starts to generate PR email threads
If you find yourself sprinkling ifs everywhere, try to lift them up, they’ll congregate at the same place eventually, so all of your variability is implemented and documented at a single place, no need to abstract anything.
It’s very useful to model your inputs and outputs precisely. Postpone figuring out unified data types as long as possible and make your programming language nice to use with that decision.
Hierarchies of classes, patterns etc are a last resort for when you’re actually sure you know what’s going on.
I’d go further and say you don’t need functions or files as long as your programming is easy to manage. The only reason why you’d need separate files is if your vcs is crippled or if you’re very sure that these datetime handlers need to be reused everywhere consistently.
Modern fullstack programming is filled with models, middleware, Controllers , views , … as if anyone needs all of that separation up front.
If your code ever has the possibility of changing, your early wins by having no abstraction are quickly paid for, with interest, as you immediately find yourself refactoring to a higher abstraction in order to reason about higher-order concepts.
In this case, the abstraction is the simplicity, for the same reason that when I submit this comment, I don't have to include a dictionary or a definition of every single word I use. There is a reason that experienced programmers reach for abstractions from the beginning, experience has taught them the benefits of doing so.
The mark of an expert is knowing the appropriate level of abstraction for each task, and when to apply specific abstractions. This is also why abstractions can sometimes feel clumsy and indirect to less experienced engineers.
Even file interfaces in most programming languages don’t come with pipelining. Most are leaky abstraction.
Most abstractions also deal with 1 thing instead of N things. There’s no popular http server that supports batch request processing.
Async-await is a plague of an abstraction.
Abstracting something like trivial if statements is not a problem. The best transaction of all, passing a function to a function is underused.
IMO a lot of (software) engineering wisdom and best practices fails in the face of business requirements and logic. In hard engineering you can push back a lot harder because it's more permanent and lives are more often on the line, but with software, it's harder to do so.
I truly believe the constraints of fast moving business and inane, non sensical requests for short term gains (to keep your product going) make it nearly impossible to do proper software engineering, and actually require these if-else nests to work properly. So much so that I think we should distinguish between software engineering and product engineering.
They fail on reality. A lot of those "best" practices assume, that someone understands the problem and knows what needs to be built. But that's never true. Building software is always an evolutionary process, it needs to change until it's right.
Try to build an side project, that doesn't accept any external requirements, just your ideas. You will see that even your own ideas and requirements shift over time, a year (or two) later your original assumptions won't be correct anymore.
The issue is when the evolution is random and rife with special cases and rules that cannot be generalized... the unknown unknowns of reality, as you say.
Then, you just gotta patch with if elses.
You’ve just described the universe. It’s full of randomness.
Add to the fact that they're the professor of many software engineering courses and you start to see why so many new grads follow SOLID so dogmatically, which leads to codebases quickly decaying.
Sometimes last mile software turns into these abstractions but often not.
I’ve worked with very smart devs that try to build these abstractions too early, and once they encounter reality you just have a more confusing version of if statement soup.
I'm not sure if that's anywhere in the rating of quality of business software. Things that matter:
1. How fast can I or someone else change it next time to fulfill the next requirements?
2. How often does it fail?
3. How much money does the code save or generate by existing.
Good architecture can affect 1 and 2 in some circumstances but not every time and most likely not forever at the rate people are starting to produce LLM garbage code. At some point we'll just compile English directly into bytecode and so architecture will matter even less. And obviously #3 matters by far the most.
It's obviously a shame for whoever appreciates the actual art / craft of building software, but that isn't really a thing that matters in business software anyway, at least for the people paying our salaries (or to the users of the software).
Certainly, there are such people who simply don't care.
However I would also say that corporations categorically create an environment where you are unable to care - consider how short software engineer tenures are! Any even somewhat stable business will likely have had 3+ generations of owner by the time you get to them. Owner 1 is the guy who wrote 80% of the code in the early days, fast and loose, and got the company to make payroll. Owner 2 was the lead of a team set up to own that service plus 8 others. Owner 3 was a lead of a sub-team that split off from that team and owns that service plus 1 other related service.
Each of these people will have different styles - owner 1 hated polymorphism and everything is component-based, owner 2 wrapped all existing logic into a state machine, owner 3 realized both were leaky abstractions and difficult to work with, so they tried to bring some semblance of a sustainable path forward to the system, but were busy with feature work. And owner 3 did not get any Handoff from person 2 because person 2 ragequit the second enough of their equity vested. And now there's you. You started about 9 months ago and know some of the jargon and where some bodies are buried. You're accountable for some amount of business impact, and generally can't just go rewrite stuff. You also have 6 other people on call for this service with you who have varying levels of familiarity with the current code. You have 2.25 years left. Good luck.
Meanwhile I've seen codebases owned by the same 2 people for over 10 years. It's night and day.
I once tried to explain to a product owner that we should be careful to document what assumptions are being made in the code, and make sure the company was okay committing to those assumptions. Things like "a single order ships to a single address" are early assumptions that can get baked into the system and can be really hard to change later, so the company should take care and make sure the assumptions the programmers are baking into the system are assumptions the company is willing to commit to.
Anyway, I tried to explain all this to the product owner, and their response was "don't assume anything". Brillant decisions like that are why they earned the big bucks.
Employees aren’t fired. They leave for a 10% increase. Employees are the ones who seek always more in a short-termist way.
They are acting rationally given companies don't seem to value long term expertise.
Now, if it was a worker- owned cooperative, that would be a different thing.
The model of having a circle of ancient greybeards in charge of carefully updating the sacred code to align with the business requirements, while it seems bizarre bordering on something out of WH40K, actually works pretty well and has worked pretty well everywhere I've encountered it.
Attempts to refactor or replace these systems with something more modern has universally been an expensive disaster.
Project Manager: "Can we ship an order to multiple addresses?"
Grey Beard: "No. We'd have to change thousands of random if-statements spread throughout the code."
Project Manager: "How long do you think that would take?"
Grey Beard: "2 years or more."
Project Manager: "Okay, we will break you down--err, I mean, we'll need to break the task down. I'll schedule long meetings until you relent and commit to a shorter time estimate."
Grey Beard eventually relents and gives a shorter time estimate for the project, and then leaves the company for another job that pays more half-way through the project.
Project Manager: "Can we ship an order to multiple addresses? We need it in 2 weeks and Grey Beard didn't want to do it"
Eager Beaver: "Sure"
if (order && items.length > 1 && ...) {
try {
const shipmentInformation = callNewModule(order, items, ...)
return shipmentInformation
} catch (err) {
// don't fail don't know if error is handled elsewhere
logger.error(err)
}
} else {
// old code by Grey Beard
}Of course Eager Beaver didn't learn from this experience because they left the company a few months ago thinking their code was AWESOME and bragging about this one nicely scalable service they made for shipping to multiple addesses.
Meanwhile Grey Beard is the one putting out the fires, knowing that any attempt to tell Project Manager "finding and preventing situations like this was the reason why I told my estimate back then" would only be received with skepticism.
/s
Your counter example assumes the people managing the code base are incompetent.
Wouldn't the rewrite fail for the exact same reason if the company only employs incompetent tech people?
I don't think it is malice or incompetence, but this happens too often to feel good.
Sales contracts with weird conditions and odd packaging and contingencies? Pile of if statements.
The other great model for business logic is a spreadsheet, which is well modeled by SQL which is a superset of spreadsheet functionality.
So piles of if’s and SQL. Yeah.
Elegant functional or OOP models are usually too rigid unless they are scaffolding to make piles of conditions and relational queries easier to run.
One would imagine by now we would have some incredibly readable logical language to use with the SQL on that context...
But instead we have people complaining that SQL is too foreign and insisting we beat it down until it becomes OOP.
To be fair, creating that language is really hard. But then, everybody seems to be focusing on destroying things more, not on constructing a good ecosystem.
The way I've been thinking about it is about organization. Organize code like we should organize our house. If you have a collection of pens, I guess you shouldn't leave them scattered everywhere and in your closet, and with your cutlery, and in the bathroom :) You should set up somewhere to keep your pens, and other utensils in a kind of neat way. You don't need to spend months setting up a super-pen-organizer that has a specially sculpted nook for your $0.50 pen that you might lose or break next week. But you make it neat enough, according to a number of factors like how likely it is to last, how stable is your setup, how frequently it is used, and so on. Organizing has several advantages: it makes it easier to find pens, shows you a breath of options quickly, keeps other places in your house tidier and so less cognitively messy as well. And it has downsides, like you need to devote a lot of time and effort, you might lose flexibility if you're too strict like maybe you've been labeling stuff in the kitchen, or doing sketches in your living room, and you need a few pens there.
I don't like the point of view that messiness (and say cognitive load) is always bad. Messiness has real advantages sometimes! It gives you freedom to be more flexible and dynamic. I think children know this when living in a strict "super-tidy" parent house :) (they'd barely get the chance to play if everything needs to be perfectly organized all the time)
I believe in real life almost every solution and problem is strongly multifactorial. It's dangerous to think a single factor, say 'cognitive load', 'don't repeat yourself', 'lesser lines of code', and so on is going to be the single important factor you should consider. Projects have time constraints, cost, need for performance; expressing programs, the study of algorithms and abstractions itself is a very rich field. But those single factors help us improve a little on one significant facet of your craft if you're mindful about it.
Another factor I think is very important as well (and maybe underestimated) is beauty. Beauty for me has two senses: one in an intuitive sense that things are 'just right' (which capture a lot of things implicitly). A second and important one I think is that working and programming, when possible, should be nice, why not. The experience of coding should be fun, feel good in various ways, etc. when possible (obviously this competes with other demands...). When I make procedural art projects, I try to make the code at least a little artistic as well as the result, I think it contributes to the result as well.
[1] a few small projects, procedural art -- and perhaps a game coming soon :)
I don't see the problem. Okay, so we need to support multiple addresses for orders. We can add a relationship table between the Orders and ShippingAddresses tables, fix the parts of the API that need it so that it still works for all existing code like before using the updated data model, then publish a v2 of the api with updated endpoints that support creating orders with multiple addresses, adding shipping addresses, whatever you need.
Now whoever is dependent on your system can update their software to use the v2 endpoints when they're ready for it. If you've been foolish enough to let other applications connect to your DB directly then those guys are going to have a bad time, might want to fix that problem first if those apps are critical. Or you could try to coordinate the fix across all of them and deploy them together with the db update.
The problems occur when people don't do things properly, we have solutions for these problems. It's just that people love taking shortcuts and that leads to a terrible system full of workarounds rather than abstractions. Abstractions are malleable, you can change them to suit your needs. Use the abstractions that work for you, change them if they don't work any more. Design the code in such a way that changing them isn't a gargantuan task.
Which items ship to each of those locations and in which quantities?
What is the status of each of those sub-orders in the fulfillment process?
Should the orders actually ship to those addresses or should the cartons just be packed and marked for those locations for cross-docking and the shipments should be split across some number of regional DC's based on proximity to the final address?
Many things need to be updated in the DB schema+code. And if you think this isn't a very good example, it's a real life example of orders for large retailers.
The details don't really matter for my main point though. The point is you don't solve this problem with workarounds. You find a way to redesign the system to suit your new needs - assuming the business thinks that's worth it. That's like our whole job, we build systems and when they need to change we change them. The only question is how we make those changes - do we do it properly or do we add workarounds until our codebase is a big pile of workarounds?
If the abstraction doesn't fit a new problem, it should be easy to reassemble the components in a different way, or use an existing abstraction and replace some components with something that fits this one problem.
The developers shouldn't be forced to use the abstractions, they should voluntarily use them because it makes it easier for them.
An underappreciated value. I call this composability and it is one of my primary software development goals.
Of course, the disadvantage is the exponential growth. 20 ifs means a million cases (usually less because the conditions aren't independent, but still).
Then I have a flat list of all possible cases, and I can reconstruct a minimal if tree if I really want to (or just keep it as a list of cases - much easier to understand that way, even if less efficient).
Making invalid data unrepresentable simplifies so much code. It's not always possible but it is way underused. You can do some of it with object encapsulation too, but simple enums with exhaustive switch statements enforced by the compiler (so if it changes you have to go handle the new case everywhere) is often the better option.
That's the dream. Error handling is what crushes it :)
It also depends how big the consequences to failure/bugs are. Sometimes bugs just aren't a huge deal, so it's a worthwhile trade-off to make development easier in change for potentially increasing the chance of them appearing.
These individual functions are easier to reason about since they have specific use cases, you don't have to remember which combinations of conditions happen together while reading the code, they simplify control flow (i.e. you don't have to hack around carrying data from one if block to the next), and it uses no "abstraction" (interfaces) just simple functions.
It's obviously a balance, you'll still have some if statements, but getting rid of mutually exclusive conditions is basically a guaranteed improvement.
I'm both bothered and intrigued by the industry returning to, what I call, "pile-of-if-statements architecture". It's really easy to think it's simple, and it's really easy to think you understand, and it's really easy to close your assigned Jira tickets; so I understand why people like it.
People get assigned a task, they look around and find a few places they think are related, then add some if-statements to the pile. Then they test; if the tests fail they add a few more if-statements. Eventually they send it to QA; if QA finds a problem, another quick if-statement will solve the problem. It's released to production, and it works for a high enough percentage of cases that the failure cases don't come to your attention. There's approximately 0% chance the code is actually correct. You just add if-statements until you asymptotically approach correctness. If you accidentally leak the personal data of millions of people, you wont be held responsible, and the cognitive load is always low.
But the thing is... I'm not sure there's a better alternative.
You can create a fancy abstraction and use a fancy architecture, but I'm not sure this actually increases the odds of the code being correct.
Especially in corporate environments--you cannot build a beautiful abstraction in most corporate environments because the owners of the business logic do not treat the business logic with enough care.
"A single order ships to a single address, keep it simple, build it, oh actually, a salesman promised a big customer, so now we need to make it so a single order can ship to multiple addresses"--you've heard something like this before, haven't you?
You can't build careful bug-free abstractions in corporate environments.
So, is pile-of-if-statements the best we can do for business software?