I would kind of appreciate having two formats, since what are the odds they would change together? While there may never be a 3rd format, a DRY importer would imply that the source generating the data is also DRY.
One of the reasons they brought me in on this project is that besides knowing how to wrangle data, I'm also an experienced pilot. So I had a good intuitive sense of the meaning and purpose of the data.
The part of the data that was identical is the description of the airspace boundaries. Pilots will recognize this as the famous "upside down wedding cake". But it's not just simple circles like a wedding cake. There are all kinds of cutouts and special cases.
Stuff like "From point A, draw an arc to point B with its center at point C. Then track the centerline of the San Seriffe River using the following list of points. Finally, from point D draw a straight line back to point A."
The FAA would be very reluctant to change this, for at least two reasons:
1. Who will provide us the budget to make these changes?
2. Who will take the heat when we break every client of this data?
That is a great way to describe it!
Of course it is all just rows in a CSV file, but yes, it is a set of instructions for how to generate a map.
In fact the pilot's maps were being drawn long before the computer era. Apparently the first FAA sectional chart was published in 1930! So the data format was derived from what must have been human-readable descriptions of what to plot on the map using a compass and straightedge.
I just remembered a quirk of the Australian airspace data. Sometimes they want you to draw a direct line from point F to point G, but there were two different kinds of straight lines. They may ask for a great circle, a straight path on the surface of the Earth. Or a rhumb line, which looks straight on a Mercator projection but is a curved path on the Earth.
You would often have some of each in the very same boundary description!
For anyone curious about this stuff, I recommend a visit to your local municipal airport and stop by the pilot shop to buy a sectional chart of your area.
https://skyvector.com is a good way to view these.
All ‘is-a so it gets this code’ models can be trivially modeled as ‘has-a so it gets this code’ patterns, which don’t have that single-use constraint… so the corollary to this rule tends towards ‘never use inheritance’.
Has-a doesn't make sense for code like the literal text reuse. It makes sense for composition and encapsulation.
Edit: I'm now realizing that Python has one of the only sane multiple inheritance implementations. It's no wonder the rest of y'all hate it.
Tho if done haphazardly by someone inexperienced, you might end up with subtle divergences that might look like they're meant to be copies, and debugging them in the future by another developer (without the history or knowledge) can get hard.
Then someone would wonder why there are these two very similar pieces of code, and mistakenly try to DRY it in the hopes of improving it, causing subtle mistakes to get introduced...
Last time i checked the FP world is slowly producing ECS frameworks that are needed to make the game performant. They used to be nearly C++ (or OO) exclusive.
Entity component systems are pretty cool, as is functional programming, but I don't see the relation.
In addition, object-oriented languages seem well-suited to making entity component systems. There are some tutorials on them in different object-oriented programming languages:
C++: https://austinmorlan.com/posts/entity_component_system/
C#: https://matthall.codes/blog/ecs/
Common Lisp: https://edoput.it/2023/11/19/data-oriented-clos.html
I agree. The primary risk of presented by DRY is tight coupling code which only bears similarities at a surface level. Starting off by explicitly keeping the externa bits separate sounds like a good way to avoid the worst tradeoff.
Nevertheless I still prefer the Write Everything Twice (WET) principle, which means mostly the same thing, but following a clear guideline: postpone all de-duplication efforts until it's either obvious there's shared code (semantics and implementation) in >2 occurrences, and always start by treating separate cases as independent cases.
But what basically ended up happening was we had 2 codebases: 1 for that non-DRY version, and then 1 for everything else. The non-DRY version limped along and no one ever wanted to work on it. The ways it did things were never updated. It was rarely improved. It was kinda left to rot.
Writing it DRY in the first place would also have costs, including the alternative costs. Would it have been better to take those there and then?
It sounds to me that you're trying to pin the blame of failing to maintain software on not following DRY, which makes no sense to me.
Advocating against mindlessly following DRY is not the same as advocating for not maintaining your software. Also, DRY does not magically earn you extra maintenance credits. In fact, it sounds to me that the bit of the code you called DRY ended up being easier to maintain because it wasn't forced to pile on abstractions needed to support the non-DRY code. If it was easy, you'd already have done it and you wouldn't be complaining about the special-purpose code you kept separated.
Undoing an abstraction is way easier. Eventually, they all turn bad anyways.
Undoing an abstraction is always far more difficult than the pains caused by not having one in the first place.
In a situation where there is a base model with specific modifications - which is, I feel, how airspace regulation mostly works - then I suspect that a DRY approach would make it easier to inspect and test, so long as it stays that way.
Was that necessarily a bad thing and something that must be corrected for that code base?
I usually follow the same rule of thumb until I find myself repeatedly updating both at the same time. If I can't update one without updating the other then they must be the same thing and its time to DRY.
Don't Repeat Yourself when updating code.
Yes, there was some ongoing maintenance of this code where both versions had to be updated. The original author was not a pilot and was unfamiliar with some of the nuances of FAA airspace. One of the reasons they brought me in was that I am a pilot and knew how the FAA's data should be interpreted.
In the end, not a huge deal, but it was annoying when I had to make the same changes in two places.
While they were wrong in this case, I would say it was a reasonable move to not DRY based on the code pattern itself at the time. And that's the big difference imo - DRYing based strictly on the structure of code vs business processes.
That's not what people want to hear - they want a silver bullet; a set of criteria for which DRY'ing could be determined from the onset!
Never zealously adopt a programming practice that can be summarized in a headline!
I've been programming for nigh on 40 years; I'm not terribly curious because I've seen successes and failures of pretty much every development paradigm firsthand. But since you ask, my comment above was a self-referential whimsicalism.
I DRY my code in phases. A rule of 3 is decent (e.g., if you've copied a chunk of code three times, give it a think; if you've copied it 9 times, give it a good hard stare).
I’ve had a lot of success with this in embedded settings where data is piped into storage or OTA, and I want to format and pack it/send it up consistently but I might want to treat the data itself slightly differently.
[0a] More generally, I like a concept of "total" functions -- those which have sensible outputs for all their inputs. It's a bit of a tomayto/tomahto situation defining "all their inputs" (e.g., I'm personally okay using a function name like `unsafe_foo` and expecting a person to read the docs, and on the other extreme some people want sensible answers to anything the type system allows you to input), but the desired end-state is that when the project's requirements change you don't muck around with the ABI and implementation of `count_or_maybe_sort_for_these_three_special_customers_or_else_hit_the_db(...)`, or whatever much more generic and very wrong name the method actually has; the individual components are already correct, so you make the changes at the few methods which are actually wrong given the new requirements.
[0b] Another way of thinking about it is whether the two things should always change in tandem. For two largely overlapping beaucratic data formats? Maybe; there's a comment somewhere in this chain suggesting that they'll never go out of sync, but I'm a bit paranoid of that sort of thing. For the particular data structures that are currently shared by those formats? Absolutely not; if one diverges then you can build the new structure and link it in. The old structure is still valid in its own right.
So if a new customer comes and need an integration to a system we already support, even if we think they'll start out being identical, we just copy the code.
Thing is, these things evolve. Suddenly we have to patch over some process-related issues in the other system for customer A, while customer B does not have that issue. Now we can fix A's integration without worrying at all about affecting B.
Of course we write library and helper functions, and use those actively throughout, so we only repeat the "top level" stuff.
3 things matter most in real estate: Location, location, location!
3 things matter most in programming: Context, context, context!
DRY – like almost every other programming tool/paradigm/principle – are very often misused by a lack of the programmer's ability to discern correctly whether the tool/paradigm/principle fits in the specific context.
It's not just a science. It's an art, too.
It was... I will miss that until retirement, too, but the artisan part of this craft has been gradually dying for over a decade. I think this change started when "popular kids" started confidently saying they wanted to work with computers when they grew up. The effect is the proliferation of normies throughout the trade, now many of them with 10+ years of experience. The average developer's appreciation for the elegant and the inspiring grew weaker, and the idea of putting more work into a task than absolutely necessary (like, for example, stopping for a moment to consider the context before deciding on a tool or technique to apply...) lost all appeal. There's a thin line between aggressively pragmatic and ignorant, and the newer generations seem to treat crossing that line as a non-issue as long as the ticket can be presented as resolved. This mindset used to be confined to cubicles and neckties, but now it's seemingly everywhere...
Don't mind me; I just feel unusually old and grumpy today...
[1]: https://mlochbaum.github.io/BQN/
The problem is, when I go to work, I see exactly none of those people among my coworkers. I feel like breaking down on the spot and declaiming:
> I've seen things you people wouldn't believe... Attack ships on fire off the shoulder of Orion... I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain... Time to die.
I don't, mostly out of respect to Rutger Hauer. Anyway, while in absolute terms people like that are probably much more numerous now than 20 years ago, they feel much more distant to me than ever before.
Also consider finding a job where you need to know category theory to understand the system! That'll give you coworkers who are magical! And a little insane!
Ps. The "C" in "C-beams" now stands for "Category Theory"!
It's best not to try! These are the cultural delusions that drive us!
If they do it, if they actually learn categories, then you need to shut the f** up and let them live. They now have the power to do what they want. If they refuse to learn categories then you get to laugh at them and tell them that they are cringe, and fire them.
And if you don't have the power to just fire them for not learning categories, then you very likely need to learn categories yourself! And also gain the power to fire people, lol! There is no rational point to ever complain about a programmer, when you can just fire them instead.
What I learned from that is to create a ticket for every slightest little thing.
Correct a typo in a comment? A ticket.
Combine a tiny bit of duplicate code? A ticket.
Add a little error check? A ticket.
Forget working together to create a great and well-supported product.
What counts is whether I closed more tickets than you!
As I recall, it was after I saw the two implementations. I got curious and looked at the original FAA data and specifications and saw how much the two kinds of airspace have in common.
We never got that far. At that point, the code was what it was. Despite any possible design flaws, it worked. And dealing with FAA data that really never changes at all, once it worked, that was good enough.
"If it ain't broke..." ;-)
My job after that was to move on to other countries' airspace and obstacle data. So I tried to apply the appropriate amount of humidity on those.
If you do what this guy did and are wrong, it looks like yes obviously it was the same thing and you should have DRY right from the start.
If you DRY from the start and are wrong, someone writes a blog post about not DRYing from the start.
It's a bet on which mistake is worse, because you will be wrong sometimes. DRYing from the start IMO is a worse mistake than duplication.
As always, "You don't know what you don't know."
One time the guy who wrote the original code asked, "Why do they have towers at airports? Aren't the controllers down underground looking at radar screens?"
It took two of us who were pilots to convince him that yes, the tower is up there so the controllers can see the airplanes with their own eyes. And yes, they are provided with binoculars.
We all form mental models of the world around us. The best we can hope for is to update those models as we receive new information.
In reality: Had he DRYed it up from the beginning, you would probably have complained about a codebase that needs to corner cases into account deep in the code - the story had been turned around.
Yes, it is annoying to get into an existing code base.
In your case I don't see how it couldn't be refactored later on once you have the domain knowledge. In our company we always have 2 types of projects planned. We can't always ship new features under type 1. And refactoring is something we must do under the 2nd type to reduce the tech debt. With 2 type of projects in mind we always plan the capacities for both types of projects
And that's how exactly how you should do dry. Once you see the repeated pattern, you should refactor it.
I'm in the finance factor and we are seeing this kind of 'similar but not the same' problem a lot. We are constantly in the process onboarding new payment use cases while doing refactors to abstract away the common patterns.
In that case, reading and maintaining two simple use cases might be less work than reading an abstract use case, backtracking to the available factories, and then mentally interpreting the injection and factory behavior.
Unless your abstractions really really make things just way simpler, being explicit could be better.
Another place where this repetition tends to help readability more than it hurts maintainability is in test cases. Often abstracting things out with a little test fixture is helpful. But then being obsessive about this ends up making tests harder to maintain since there's all this long distance coupling that you constantly have to maintain.
It seems that outside of test cases and use cases, we want to be much more diligent about DRY and picking the right abstractions - that makes logic in our use cases much simpler and more coherent. While inside the use case or test case, a little duplication of business logic is not so bad, and can actually improve the narrative of the code.
The day one of those formats changes slightly, it's really easy to update it. If you had a single code base, you now have to think how to refactor out the differences. In my experience when those differences appear is more likely to not be trivial.
Several years ago, I did some contract work for a company that needed importers for airspace data and various other kinds of data relevant to flying.
In the US, the Federal Aviation Administration (FAA) publishes datasets for several kinds of airspace data. Two of them are called "Class Airspace" and "Special Use Airspace".
The guy who wrote the original importers for these treated them as completely separate and unrelated data. He used an internal generic tool to convert the FAA data for each kind of airspace into a format used within the company, and then wrote separate C++ code, thousands of lines of code each.
Thing is, the data for these two kinds of airspace is mostly identical. You could process it all with one common codebase, with separate code for only the 10% of the data that is different between the two formats.
When I asked him about this, he said, "I have this philosophy that says if you only have two similar things, it's best to write separate code for each. Once you get to a third, then you can think about refactoring and making some common code."
That is a good philosophy! I have often followed it myself.
But in this case, it was obvious that the two data formats were mostly the same, and there was never going to be a third kind of almost-identical airspace, only the two. So we had twice the code we needed.