The best codebases I’ve seen have an abstraction layer at the foundation and the rest is just dumb repetitive code built on top of that layer.
It’s tempting to look at that and think we can abstract it out a bit more, but then you cross the line pretty quickly and your abstractions now mean you have 20 parameters to control how the abstraction should work.
That's not to say I haven't had to replace data storage, but they were never built as layers and inevitably it happens when moving from mainframes to more modern systems. At this point at least some of my abstraction feels like a waste of time. Maybe when we start moving to Quantum Databases someone will benefit from my extra due diligence from a decade ago.
When you have a 'system' which is 500 LOC, and 10 usages of that system totaling 10 lines each; compared to 10 bespoke implementations of 50-100 LOC each... you get into a spot where the system is battlehardened because of all the users, while the bespoke implementation have issues all over. Then when someone goes to add the 11th use-case of this; they are going to learn from and copy/paste the other version, probably exposing some latent bug in the bespoke version.
It's okay to have someone else manufacture the bricks for you. However, you still have to lay them by hand. It's tedious, repetitive, and labor intensive, but gets you the exact result you want.
The second you try to pre-cast bricks, you end up with something harder to work with.
On the other hand my hacky script, put together on a sunny summer afternoon, has zero love from senior management as they probably don't even know it exists.
For me, I follow the principle of three: if it's repeated three times with minor tweaks, I should probably make some kind of abstraction for it. I try not to start with abstractions, as they tend to distract from solving the initial problem.
Eskil Steenberg has talked about *width* of code as a more likely quality indicator.
Consider the difference between:
func(int i, void* p)
And: modulename_objectname_operation(int count, void *ptr)
Reference: https://youtube.com/watch?v=443UNeGrFoMI'm sorry, nothing intelligent can come after statement like that, and I'm not wasting 2 hours to verify that claim
Everything in software is an abstraction. Source code is an abstract representation of the running program. Functions are abstract representations of various execution that differ only in their parameters. Classes are abstract representations of stateful concepts of various instantiations that vary only in the information each instance contains. Etc.
You cannot "not use" abstractions when you program, you can simply design poor abstractions or design good abstractions. And a lot of people underestimate the second degree impact of even the smallest of abstraction design choices, like what is and isn't factored into its own function, what parameters the function takes and return, what classes we have, how big they become, etc.
And while saying no to new use cases that are almost same but slightly different is a good way to avoid needing to abstract more things (or avoid needing more code in general), and good engineers push back when necessary here, you will eventually have to deliver what the users want, and you cannot always convince them that all users want the same exact behavior as all other users, there will sooner or later be good business justifications for it.
That's when if you don't introduce good abstractions to tackle the problem, you'll end up in an unmaintainable mess which will turn your velocity to a crawl.
But it's important to understand abstractions have to be designed in tandem between product and tech, because how the user feature is exposed is also an abstraction, both the feature abstraction and the tech abstractions have to be well designed and must help each other out to remain manageable and coherent and work well with other features.
In my opinion this is where really great engineers distinguish themselves, it's the ones who can figure out the good way to abstract that delivers that business international expansion where each region has slight variations yet feature parity can be maintained accross region, as an example of when you won't be able to push back and the abstractions will become critical to success. Normal engineers brute force their way through this slowly, each new region taking longer to expand too then the previous took, and with new features being impossible to launch globally anymore. While great engineers figure out the right set of abstractions alongside the product to enable fast expansions with judicious configuration/plugins to handle per-region variations, etc. They invest in tooling, frameworks and platforms that create those kind of abstractions, and just have the good instinct around balancing tradeoffs to deliver simple yet powerful abstractions that actually makes you faster and the system more maintainable.
Bottom line, I find abstractions are critical to success, saying to avoid them I believe is misleading, you cannot avoid them, you need to get good at them instead.
This is one of the hardest lessons I've had to learn as an engineer. Building extensible abstractions upfront is extremely difficult to achieve without building a lot of abstractions upfront that aren't extensible, and learning why.
> you cannot avoid them, you need to get good at them instead
I'm working on it!
While I agree with the comment as a whole, I think a part of the argument is about not introducing unnecessary abstractions which might couple code that might be better off remaining separate.
As another commenter put it:
> The best codebases I’ve seen have an abstraction layer at the foundation and the rest is just dumb repetitive code built on top of that layer.
https://news.ycombinator.com/item?id=33569197
Suppose that you're building a system for supporting a sales process and keeping track of bunches of data. And based on some BPMN diagrams, you might have multiple different sales processes to keep track of: A, B, C. At a surface level, all of them deal with some company receiving money in exchange for goods and services, but the particulars of each process (the steps involved, the documents, the state transitions etc.) are going to be different. What's worse, they're going to change with time and evolve separately.
So, the aforementioned and boring architecture would be a bit like (which might get created across 5 years, as new processes need to be supported):
SalesProcessAResource/View (REST/UI) <--> SalesProcessAService <--> SalesProcessARepository <--> sales_process_a_schema
SalesProcessBResource/View (REST/UI) <--> SalesProcessBService <--> SalesProcessBRepository <--> sales_process_b_schema
SalesProcessCResource/View (REST/UI) <--> SalesProcessCService <--> SalesProcessCRepository <--> sales_process_c_schema
Each process might have a separate schema with different tables, different services and repositories and resources for them (not unlike microservices, but assume that this is within a single codebase, maybe modules). There would certainly be overlap which wouldn't be ideal and in the first iterations those might seem somewhat similar/redundant, but this way you can evolve each part separately.For example, if your sales process A now needs to have products have groups in front of them, or even products that can only be bought as package deals/sets, then you can change all of the layers:
SalesProcessAResource/View (REST/UI) <--> SalesProcessAService <--> SalesProcessARepository <--> sales_process_a_schema
And then there is basically no risk of you ruining or breaking the other sales processes, because they're all decoupled and are handled differently. Now, the alternative to something like that is: SalesProcessCommonResource/View (REST/UI) <--> SalesProcessCommonService <--> SalesProcessCommonRepository <--> sales_process_common_schema
I've often seen is developers trying to create "the one solution to rule them all", which leads to code that's really abstracted, maybe the SalesProcessCommonService has some abstract methods that are implemented by separate services, which makes your code paths harder to track, maybe there are now enums all over the place or your code is full of if/else or switch constructions.Worse yet, your schema might look like swiss cheese, some of the columns always being empty because the data normalization isn't up to par, or perhaps going in the opposite direction and needing to query 10 different tables to get the data that you need, or worse yet "classifier systems" that I've seen a lot of in my country, basically the worst of OTLT/EAV patterns: https://tonyandrews.blogspot.com/2004/10/otlt-and-eav-two-bi...
I've personally seen many similar outcomes in projects, all the way to needing to introduce some data in the middle for one business case, and creating faux records there because the schema doesn't work otherwise, for example:
SELECT
...
FROM sales_processes
INNER JOIN sales_sets
ON sales_processes.id = sales_sets.sales_process_id
INNER JOIN sales_products
ON sales_sets.id = sales_products.sales_set_id
AND sales_sets.code = 'NO_GROUP_FOR_PROCESS_ONLY_LINK'
WHERE
sales_processes.code = 'SALES_PROCESS_A'
or something like that.Of course, one can come up with the right abstractions for something like that, or extract common parts of code, but only as long as they're not logically decoupled and the code is only incidentally the same (or close to it).
Also, in some cases you might need a time machine to see what the requirements will be in a few years, otherwise you risk digging yourself a hole by over-abstracting things and going off in the wrong direction, to the point where you'll have thousands of person-hours sunk into an inadequate implementation, or at least one that becomes inadequate in the light of changing requirements.
That said, there are definitely easy cases where abstractions are of use without giving it too much thought. Think along the lines of code that is either purely functional or close to it, "utilities" that you can use as necessary with simple function calls or maybe injected dependencies, versus something that makes you restructure your entire code because someone went on a power trip after reading an OOP book.
My guess is most devs learn this the hard way. You are liable for bugs in code you write, regardless of project quality and how much you complain to management about it. Apart from the examples you describe, I'd like to also emphasize cognitive load VS size/complexity. After a certain threshold, you shouldn't add more features but become a maintainer/tester of the ones you previously added (until you let go of responsibility/ownership).
Oof. I work in finance and if you say that to a trader/business manager you'll be out of a job. Rightfully so.
> And you might argue, "well why don't you abstract more".
A place I work was acquired by a company that owned our main competitor. When we were comparing them, I used to joke that the team had spent 10 years saying "yes!" to literally every feature request.
They of course did it with abstractions, so it "wasn't that much more complicated".
Feature-for-feature, our software could only maybe 15% of what theirs did, while theirs could do 95% of ours. The difference was the features we picked were the only ones 80% of the customer base actually used.
Worse, because everything was abstracted, setting up those simple use cases was usually an order of magnitude more complex for the users: something that was a checkbox in ours was 15 different toggles and inputs, and very easy to screw up. Sure it could also be setup to handle hunderds more scenarios.. but in reality only there were just a few that only a tiny fraction of users even cared about.
That, to me, is the danger of abstractions - at least it they make it up to the UI layer. QA (and automated testing, if it was written) still has to test the hunderds of scenarios because they're possible. Support has to understand. Dev has to maintain.
It was interesting to compare the approaches, at least, and I came away with the conclusion: I never want to build that way. I'm all for internal abstraction: build the capability and flexibility to expand in, but have a very concrete and constrained UI that only exposes what actually has real value. And make it dirt simple to do the most popular use cases.
Exposing more switches is the opposite of an abstraction.
> , but have a very concrete and constrained UI that only exposes what actually has real value.
This is an abstraction.
Fully agree. In addition, I find there's also the time component. Because requirements for existing functionality change over time, both code and data become inaccurate and a mess if they are not refactored. This leads to more bugs and to less trust in the data model (e.g. when can something be null).
I'm not sure what domain you work in, but in mine, the feature set is determined by the business requirements. e.g., if you're writing an app, you can't say to the client, "Don't add this potentially revenue-generating feature because it'll introduce bugs." What the article is suggesting, is that writing the same feature set with less code is likely to lead to a more correct implementation.
[1] admittedly in some cases you can't for various reasons, including bosses that won't listen.
With that said, many requirements are technical and then you've even bigger freedom.
At the end of the day, you answer to the bottom line. If you can achieve what is needed of your program (i.e., the business requirements) with less code, it seems you're less likely to have as many bugs - the point of the article. I think here, a good choice of expressive language/framework can pay dividends, moreso than trifling over the product owner over what functionality should go.
If you say "You've projected so-and-so much revenue from this feature. Our projection is that it's implementation will cost this, future maintenance that, and it will introduce bugs which will lose you this much revenue and cost this much to fix. We propose you change the spec this and that way, and you will get out of most of the drawbacks while still retaining most of the benefit"
any sane business would not go "shut up and go home" but rather engage "okay, but what about sub-requirement xyz -- did you consider that in detail?" and from there on you can create something amazeballs together.
I think as an engineer it's your job to do a bit of cost / benefit analysis on what you're being asked to build and at a minimum provide a rough estimate about the time / effort it will take to build and maintain so everyone is on the same page. And sometimes I think it is your job to argue that something shouldn't be built at all.
I guess a good example of this would be at the place I worked a few years back... When I first got there one of the complains I would hear a lot from the business was that the site was too slow and that every time they asked the dev team to improve it they only managed to make limited improvements.
So I took a look into it and realised that the main issue was that we had something like 20 marketing scripts running on the site. When I started asking why we have [x] script someone would always provide me a valid reason, "Oh, Sally and her team use that for [y] reason". Okay fine, so then I ask Sally if we can remove it, but she obviously says no because this specific tool best suites her team's needs.
You could argue no single dev was really to blame here. At each step along the way they were just adding a single script to fulfil some business requirement, and as a team the devs couldn't reduce the number of scripts to improve performance because they were all deemed essential by the business.
But I think this is exactly where a good engineer would start pushing back against the business. Given this was a large corporate I spent a few weeks escalating the issue higher up the business until I finally got authority to start removing scripts and was able to put in a process for requesting new ones to be added. Imo the main issue here wasn't even the scripts themselves but that the dev team just didn't have enough authority about what they were being asked to build so they'd knowingly build crap because was what the business was asking of them.
Me pushing to remove the majority of these scripts and give the dev team more authority pissed a lot of people off. Several teams were arguing that it was going to have an impact on their performance, and in some cases that they couldn't do their job at all. Higher up in the business people were pissed because now the dev team started saying no to them for the first time ever, but overtime the benefits of this became obvious to most people. Site performance improved dramatically, the dev team were spending way less time fixing bugs, we were able to prioritise dev tasks like upgrades, etc. And eventually a few people even thanked us lol.
I guess I'd never actually say, "no I'm not building that at all" - this is just how I feel a lot of the time. I think it's more about explaining costs and trying to get everyone onboard that something shouldn't be built when it doesn't make sense.
> if you're writing an app, you can't say to the client, "Don't add this potentially revenue-generating feature because it'll introduce bugs."
See this actually worries me - why do you feel like that? If you think what you're being asked to do could introduce bugs and you're not convinced about how much revenue it will generate then you probably should be having a conservation with the client about whether it should be built in the first place imo.
I feel like I'm repeating myself a lot so I'll point you to another comment:
https://news.ycombinator.com/item?id=33573035
> And to my original point: that a business requirement adds more potential bugs is not sufficient to obviate implementing it. The only sufficient argument is that it doesn’t provide the product owner as much value as it costs to implement and maintain, but rarely is the developer in a position to judge that - they just provide the cost estimates and the PO can decide if they want to wear it or not.
It's more an issue of competent estimation and communication than gatekeeping. If the complexity costs have been adequately communicated by the technical staff to the product owners/managers/stakeholders/etc, and they insist on decisions that both add significant maintenance cost and, by your humble measure, aren't positioning the product better, it's time for you to move on to different company/client.
> So I took a look into it and realised that the main issue was that we had something like 20 marketing scripts running on the site. When I started asking why we have [x] script someone would always provide me a valid reason, "Oh, Sally and her team use that for [y] reason". Okay fine, so then I ask Sally if we can remove it, but she obviously says no because this specific tool best suites her team's needs.
Not sure this is in the scope of what we're talking about. Intra-organisation conflicts on the business requirements don't mean that business requirements don't have primacy here, it just means those requirements were badly specified and badly maintained. Perhaps a faster website is a greater requirement than the scripts were for the marketing department, but the business requirements for marketing haven't gone away, and really an alternative and performant system should have been made in their stead to enable marketing to do their job well again, again, if its decided that this provides values for the business. And the developer doesn't really have the authority to speak on whether it does or not.
On a technical level, I fully agree with this. Someone recently asked me why I didn't use an overengineered abstraction that they made for a feature request, which was not only complex, but also had no documentation or other usage examples. Actually even discovering that it was there wasn't something you'd naturally do when working on that codebase, given that the discoverability isn't exactly excellent and people don't communicate that much.
Explaining to them that using it would have slowed me down, just how working with some other of their abstractions in the past has, was an uphill battle (and many of which weren't precisely the right abstractions and needed further refactoring to be useful), despite the fact that the current implementation without the abstraction does what's necessary and works without issues in prod, as well as is simple to refactor/throw away when necessary.
In the end I simply conceded to do some refactoring and use their abstraction... but probably much later, since I have other things to do. I find myself caring less about the codebase and intricate mechanisms in it with time, because on a technical level it's boring and there really is little to no value added by bunches of complexity in it.
Of course, there are a few cases where abstractions make sense, but in my experience developers (myself included) love to overengineer things and feel clever - they build their own castles atop big balls of mud.
> A good engineer in my opinion isn't one who can build things well, but one who is willing to say, "no I'm not building that at all" and even remove features that are being used if the cost of maintaining those feature are not worth the benefits they provide.
In regards to features, however, in many environments out there you won't get the ability to say "no".
Someone who uses whatever the system you're developing is will have some requirements. A business analyst will turn those into a user story with acceptance criteria. You'll have to implement everything for it to be considered resolved, otherwise QA will throw the ticket back at you and indicate that some of the requirements haven't been met.
I'm not saying that that's how we should build software, but rather that in many places that is simply the reality of building software and it's not like you can always quit and work elsewhere, when you need the money (not every place is as well paid as the US) or when the development culture is like that in the entirety of your country/time zone.
How different is it, really? Any time I've done the research, it turns out line count correlates pretty strongly with feature complexity. Not perfectly, of course, but given how cheap line count is to produce compared to a quantification of feature complexity, it's a pretty damn good metric in my book.
(Fun thing you can do: first try to estimate how much a software solution you're intimately familiar with has grown in feature complexity the past year (about 60 % for one project I work with), then look at how much it's line count has increased the same period (exactly 79 %). Previous year I had a project with a 30 % × 15 % pair. The numbers won't match exactly, but they will most likely be in the ballpark. You won't find a project with no complexity growth double its line count, or vice versa.)
For obvious reasons, number of features correlate with size, but I also noticed that bad code tends to be too big for the little it does. Dead code, repetition, rewriting functions that are already available because you don't know your API, etc...
Note that it is a metric, not a goal. Making your code smaller won't necessarily make it better (ex: minification), but big code is more likely to be bad and successful efforts to make it better will most likely make it smaller.
It's generally a good thing to write code which takes up more space if it's clearer to understand what's going on. However the size of a codebase in terms of its features and components is almost always directly correlated to the numbers of bugs you're likely to find and the difficulty you'll have maintaining it.
A problem I have at my current place is that the dev team in my opinion is far too happy to add new features whenever they're requested. You know those kinds of requests that go something like, "I basically want something that does exactly the same as [x] expect with this one small difference". When that happens few times it's fine, but when it happens 100 times suddenly you're managing a code base which is many times more complex than it needs to be with very marginal benefits.
And you might argue, "well why don't you abstract more". And that's a lesson that took me about a decade of software engineering to learn. If you can avoid abstraction you should. Writing, $a = 2 + 2; is far easier to understand than, $a = math.sum($num1, $config->get('num2')); The need for abstraction should be seen as a code smell. That's not to say you shouldn't abstract when it's the right thing to do, but if you find yourself needing to create abstraction on top of abstraction your feature set is probably needless complex.
A good engineer in my opinion isn't one who can build things well, but one who is willing to say, "no I'm not building that at all" and even remove features that are being used if the cost of maintaining those feature are not worth the benefits they provide.