You're 100% correct in this. And what's even more amazing to me is that even after you explicitly calling this out, the majority of people replying to you (and presumably have read the article) still think the problem is between 2 & 3.
The argument she is making is not "don't make abstractions until you're 100% certain they are correct". She is essentially saying make abstractions where appropriate. Some of these abstractions will be wrong. When you start seeing yourself making certain behaviors it's probably because it's the wrong abstraction, so back it out and refactor.
Ultimately that abstraction seemed right based on the info known at the time it was created, now that you know more don't try to cling to it because it was already made. Be ok with backing it out and refactoring.
In my opinion this distinction is applicable and thus useful in contrast to whining about leaky abstractions: http://beza1e1.tuxen.de/leaky_abstractions.html
It just so happens that complecting several slightly different uses in one abstraction comes at a significant cost. Backing out (inlining the abstraction, eliminating unused code) is a simple recipe to let you see the true amount of overlap, which may or may not itself be a suitable candidate for a smaller abstraction.
When you first modify the abstraction, it's nearly perfect. Just one tiny conditional and it's a perfectly suitable abstraction again. The problem is, when this process repeats itself, you slowly get to the point where any one client of the abstraction is only using a small fraction of it, but there was never a singular point where someone made a decision to use the abstraction when it was anything less than "almost perfect".
Causes of this I've seen:
- sunk cost fallacy
- not spending enough time to understand the codes behaviour (and its side effects)
- code being written so it becomes hard to understand its behaviour and any side effects.
- trying to "win the PR game" by avoiding refactoring
- giving up on refactoring because thinking is too hard
In day to day life as an engineer, I find that we have very few _enduring_ abstractions - there are very deep ones, like the concept of streams, things like filesystems and related ideas, the concept of a virtual machine in the process sense, and so on - and a lot of faddish abstractions that have a pretty wide blast radius when they start to go wrong. A lot of the good ones (networking has a _lot_ of these, such as the abstractions above and below the model of an interface in professionally-written network device code) are focused on layering.
But in my daily work, I routinely see abstractions just come up very naturally all the time. Sometimes they turn out to be slightly wrong, but often also not. I need to perform some calculation (e.g. for billing)? That can be abstracted away. I need to parse some unstructured user request into something structured? That's an abstraction. And so on. A lot of these things are clear to me even before I start writing code.
I also tend to use (at least some amount of) DDD, to write small, composable functions with few side effects and to be as declarative as possible. All of this might help with coming up with lasting abstractions.
But I'm not denying that I'm running into lots of situations in my daily work where it turns out some abstraction was wrong. Just that I find that many more of them actually turn out to be correct or at least correct for the most part (it might be that something needs to receive an additional parameter or to returns some slightly different structure to account for error conditions or so, but it's still basically the same abstraction).
Abstraction is just hiding some complexity in implementation details behind a simpler interface. It offers benefits from reducing the need to deal with the full complexity everywhere else. It also has costs. The interface establishes a new concept, albeit a simpler one, that must also be understood and maintained wherever client code uses the abstraction. Moreover, if you need to understand or modify the detailed implementation later, there is now a barrier to doing so.
When we define an abstraction, hopefully we do so because the benefits outweigh the costs at that time. The simpler the interface relative to the complexity of the implementation it hides, the more likely this is to be true. However, that balance is inevitably subject to change as a program evolves and the relevance of the hidden details to different parts of the system changes.
So it feels like abstractions might be better characterised by whether they represent good value under the current circumstances. It is perfectly reasonable for an abstraction to be cost-effective at the time it is added, but to become more or less so as the context evolves. If it reaches a point where it is no longer cost-effective, it should be removed. Either the relevant parts of its implementation can then be inlined at each place that previously used it, or some new abstraction(s) can be defined that better reflect the relevance of different implementation details at that time.
I was under the impression this was about generalizing the abstraction to something reusable.
I.e. calculate_vat will always be used by the same thing and there will only ever be one of them.
The generalization would be:
interface SalesTaxCalculator {
func calculate_sales_tax
}
This kind of abstraction is more uncommon.What im guessing the article is struggling with is that making abstractions like salestax calculation os usually kind of useless, there is only 1-3 types of sales tax calculation and its best defined inline next to the proce calculation or put in a function like calculate_vat.
A lot of developers use abstractions to remove code duplication.
Its more fun using this kind of code than code where devs dont try to remove duplicstion but i just wish instead of seeing duplication as evil, they instead saw it a sign that their design is wrong.
If youre seeing duplication, there is an abstraction there, whether it is worth it to implement is usually more to do with if its just hiding code duplication or if it is creating a new concept that hides implementation details.
But I think there are lots of abstractions often missing from code that I have seen that don't need any sort of polymorphism. A common example is performing input validation all over the place instead of at the boundary with some sort of form object that only accepts valid params.
Input validation, and any abstraction that smells like a framework or library make great code folds for abstraction.
The question is all about cost. Does a single layer abstraction, at the boundaries, with clear inputs and outputs have much of a cost?
I don't think so.
Does forcing vat calculation into a new class far away from the widget price class have a cost? I think so, the cost is having the same concern separated for no material gain other than perhaps reducing one form of duplication for the cost of creating another.
I think you're right that it may not be necessary to extract "calculate_vat" into another class. It just really depends: if the VAT code is so complex that it should be properly unit tested, I think it deserves its own class. If it's fairly straightforward and only used inside of a single other price calculation class, then yeah, it can remain a private method or so.
But I don't think this is a question of whether to abstract or not, rather about what the right level of cohesion and coupling is for a given situation.
The article agrees that the abstraction was useful when Programmer A wrote it, but it remains vague about whether Programmer A's judgement was correct at step 2 or 3!
I'm not quite sure what Sandi Metz is thinking right now, but the original 2014 talk[1] linked in the article is pretty clear about _keeping the duplication_, and _waiting for the right abstraction_. This statement sounds more in line with the Rule of Three, or the Go proverb 'a little copying is better than a little dependency', which is what people are arguing here.
Personally, I believe that any non-trivial duplication should be eliminated at the very first chance, to avoid unintended business logic divergence. I find the later case more risky than having to deal with the wrong abstraction, since - as Sandi eloquently explains here - wrong abstractions can (and should!) be refactored.
Which emphasizes the OP’s point that the article isn’t about what happens at steps 2 or 3 at all. So it’s not concerned with whether the judgment there was right or wrong.
I can see how this could work in some cases, but I remain skeptical about this technique. I have a strong gut feeling that over 20% of the junior engineers touching this code will not understand the dup tag and will end up modifying one case without keeping the other case in sync. This number is already high enough to be worrisome for me.
It depends of course, but I personally feel the work of 'simplifying an abstraction', is easier than the problem of 'tracking down anything that might need to be edited'.
If two blocks of code refer to exactly the same thing (event, process, object, rule) in the application domain, then the duplication can be eliminated.
You can't eliminate duplication without asking, "what does this code mean in this context?".
One is the use of code quality tools like CodeClimate. It's true that those can sometimes be extremely aggressive when it comes to duplicate code to the point that I find their complaints to be uselessely beside the point. This is especially true if you have typical "structural" duplication like "many controllers start with the same sequence of steps" etc. or is even worse when you have to use configuration DSLs etc.
OTOH, it has been my personal experience that many people, if they use CodeClimate etc., routinely just ignore them for the most part, so I'm not always sure what the point of them is. But maybe other people have different experiences and some people really are routinely overabstracting the most coincidental of duplication in which case I agree that that is not a very useful thing to do.
As for the advice itself: it is definitely problematic if it is used as some sort of hard "rule". If it is taken as a heuristic / "rule of thumb", then it might be ok as long as you make sure people don't overemphasise it where other/better rules of thumb would be appropriate.
For example, if I were to write some billing code and somebody else just duplicated that code somewhere instead of using a shared abstraction, I would probably find that to be a serious code health issue as you really shouldn't perform billing calculations in two separate places: this is something that needs to be kept in sync across the code base; one sibling (nephew?) comment is right in pointing out that here you have to consider the cost of things that need to stay in sync accidentally going out of sync.
There are many more examples which is why I think that if you use "refactor on 3" as _one_ heuristic, it's fine, but if it's the sole one, then less so.
Absolutes like that are rarely good advice.
That's very generally. You might want to create abstractions before then, but be prepared that they will be wrong, and don't invest in e.g. lots of unit tests, because when you break the abstraction you'll throw away that work. Some unit tests yes, but more in semi-integration tests that verify the stack sandwiching the under-proven abstraction.
I think the article has the premise that once you remove duplication you have committed to using that abstraction and need to shoehorn all future changes into that function. That’s a ridiculous notion. You can simply refactor it again if the abstraction no longer makes sense.
I don’t understand how you read this. The premise is exactly the opposite...
I often consider "is this abstraction going to be prone to misuse" (regardless of if it's the second, third, fourth... copy) and try to head it off with either strict typing or comments or internal visibility - to try to do step 3 without opening up as big of a door to step 6, but the important thing is less when to do stuff like this but just to try to avoid things reaching step 7.
But the headline does.
I had to read quite a long way down the page to discover that all she is advocating is what i have always done: deduplicate when practical, undo the duplication when new requirements make it incorrect and push the unique parts into the callers.
That’s not really fair, it repeatedly says “wrong abstraction”, in the title and in the article. At steps 2 and 3 it is still the right abstraction, duplication only becomes better when it is the wrong abstraction.
Sandi Metz IMHO doesn't claim that the problem occurs at step 2 or 3. She doesn't claim that it's wrong to introduce abstraction when there is duplication.
What she is saying instead is that the problem occurs from step 6 onwards: when you find yourself wanting to reuse an abstraction that, regardless of whether it made sense in the first place or not, has outlived its usefulness.
I think this is in agreement with other points that she often makes, about being bold, but methodical about refactorings.
The whole discussion about "you should never abstract away code before you see the third duplication" has little to do with the article, and I'm also really not sure it's good advice.