For a project we had to write a program that differentiates mathematical equations.
I dove in and just started writing code. Eventually, I realized that I had made a design error and that my code was much more complicated and cumbersome than it needed to be and I was getting stuck due to the complexity of the monstrosity I created.
Unfortunately I figured this fundamental design flaw the night before we had to hand the assignment in and there was no way I could rewrite everything from scratch. I had to push through and put lipstick on this pig as best I could.
In the end, I had spent way more time working on this project than my friends who had spent time upfront designing their programs instead of just jumping in.
This has taught me the lesson to always first try and think things through and come up with some kind of initial design, instead of just jumping in and writing code blindly. Yes you can always refactor, but some early design mistakes can cost you a lot of time, and perhaps make refactoring unfeasible compared to a complete rewrite.
As they say "measure twice, cut once".
If you could delete half your house and re-build it at zero cost, it might be more valuable to just go for the first attempt and learn from it rather than trying to do everything in theory up front.
If you find yourself working on a "monstrosity" maybe you haven't seen the signs soon enough that you need to take a step back and refactor. But in my experience, at least starting on the problem with a POC gives you so much more high quality information that even if you have to scrap and re-write part way in, you're going to reach such a better result than if you try to map the whole thing out first without actually having tried to solve the problem.
I have made similar avoidable mistakes of not thinking it through enough, could have saved me a lot of rewriting, which was pretty expensive
Again to the house analogy, if you could just build 3 vestibules to see how they fit with just a little typing, that would be far and away preferable to committing to everything on paper before hand.
Example:
Dendrite[0] was going to be a Rewrite of Synapse (which was a prototype which ended up going into production). The rewrite started more than 5 years ago, had lots of development breaks and it still is nowhere complete or close to replace a existing Synapse instance (which even today is ... Well ... suboptimal software).
The current plans are to support and use both servers long term, because Synapse is already too widespread.
I wouldn't call that cheap.
Isn't your example proof of the benefit of pragmatic coding? Synapse is actually serving users right now. Dendrite sounds like a "better design" which is stuck in purgatory.
https://matrix.org/blog/2020/10/08/dendrite-is-entering-beta
You don't see all the perfectly designed codebases that never went anywhere.
We only talk about legacy code because it's the one that pays the bills.
My definition of "good-quality" code is pretty much exactly "how difficult would this codebase be for a new engineer to understand and modify safely."
Often the worst code comes from prolific people. There’s just so much if it. And if you touch it you will break it at least 1% of the time, so you have to pick your battles when you are trying to keep the ratio under control.
For everyone else this is tautological. Good code can continue to be good code.
People who only have green field projects as their context are very frustrating in conversations like this. They make suggestions like, well, don’t fuck up in the first place. I don’t know what your history is but that’s the feeling I’m getting.
I could assume you're throwing shade on "prolific programmers" out of some sense of insecurity, but it wouldn't be fair to generalize about strangers on the internet ;)
I just punch up at condescending people. Every discipline has a bunch of armchair people who don’t understand the problem who think “get more exercise” is the response to depressed people or people with chronic fatigue, “eat fewer calories” is the answer to weight issues or diabetes, or “write it right the first time” is a useful response to people trying to solve real world problems.
Kindly let the grownups talk and keep your flash card answers to yourself.
Well one of us is certainly being condescending ;)
And did you really answer twice to the same comment? My goodness I must have really struck a nerve.
Code has a lot of really great properties which make it easy to modify in provably safe ways if you know what you're doing.
Which isn't to say that rewriting can't be cheap, but some intentional design (or at least diligent maintenance and refactoring) must have gone into the system to support that style of development. At which point you're back to targeting "quality," even if it's no longer a focus of on the smaller scale.
If the amount of time that it takes to rewrite something is trivial (a week or less) then you aren't working on something all that big/complicated.
If you're writing a function implementation without thinking about design, well you might be right.
Maybe some people have very small projects compared to what I work on? Or maybe they are talking about the design of a single small component?
I inherited a codebase that needed some refactoring because it was written “to just get it shipped”. If completely fell apart with more users and has taken me a year to get it where it needs to be.
As you need to add functionality, you can opportunistically find ways to improve the codebase toward a better/more maintainable structure.
I have several large codebases I work on which probably get totally rewritten every 18 months or 2 years, however incrementally.
If you have a working project and you think you need to throw it all out and start from scratch probably it's worth a second thought.
The second elephant in the room is job security. People who write baroque code are hard to fire. Nobody wants to invest energy in understanding their private little Bedlam.
It is also notoriously difficult to get the design of concurrency primitives correct without thinking things through.
I have no idea why there is a large (or at least vocal) community of developers whose dogma seems to be that thinking things through is a waste of time (though maybe it is just an overreaction to the equally dogmatic clean coders and similar prescriptivists.)
My point is more that coding itself is an excellent tool for probing for solutions. In many cases I think "software design" is over valued, and time spent prototyping is often more valuable than time spent thinking though the problem if you want to arrive at a high quality answer.
The hard part is done, but basically we had to switch the engine while the car was running so to speak. In order to switch this out you need to start writing the correct data shape, then switch everything over to reading that shape. When people work strict 9 to 5s this will take you forever especially when managing a large amount of volume, which requires you to be extremely risk-averse and slow.
If I have to rewrite a bit of my code then them’s the breaks. But I work on a team, sometimes a big team. I don’t have “a house” I have a construction crew that is building many houses and will go on building them. If they’re doing it wrong then I have not only the problem in front of me but five copies elsewhere. And I can’t fix problems N times faster than they are made. And I can’t always sell them on the better technique, even when there are demonstrable problems with theirs.
And of course I don't mean that you should check in code which is a mess. But from the time you start a feature to the time you open a PR, you can go through several iterations of messy code before arriving at a solution which is fit to share with your colleagues.
In a couple of notable cases, that didn’t stop until I removed the last copy. My theory is that certain people were cutting and pasting code from one of the three surviving copies.
I look forward to that day when atoms are commanded like bits!
Top down and bottom up architecture have their places. Being extreme in favor of one side is usually bad, as almost anything in life.
There is no incrementel rewrite between different stack/heap handling as those are an absolutely central parts of the design, which are pretty much impossible to try to encapsulate, as opposed to the 34th API endpoint. So what it means is that certain domains have much higher essential complexity and at that point the average encapsulation given by OOP/language tools are not sufficient to contain these parts, complexity will triumph and the whole program has to be viewed as one unified whole. Concurrent applications are a similar can of worms.
I have applied incremental design to concurrent applications and a compiler + stack VM project that runs in embedded environments. You don't go in blind. You do need domain experience and broad strokes knowledge of the conventions. You make some major architectural decisions up front but these don't involve much planning or design. Contrary to your point about CRUD apps, API design is harder to achieve incrementally since it is an interface and requires cross-team (sometimes cross-organizational) iteration. It's still possible, but your organization needs to be equipped for incremental/agile work.
But it's interesting you mentioned compilers - I'm in the process of writing one, and I very much used an incremental approach.
The first pass I essentially wrote a parser and a component which walked the AST and produced output. At a certain point it was clear that local knowledge of the AST wasn't sufficient to capture non-local details about the program which were required to produce the correct output. I got away for a short time with dirty tricks, but eventually transitioned to a new design: I kept the lexer and parser, and implemented a data driven IR in the style of an ECS system to be built up before emitting output.
So I threw out the initial output component, but I learned a ton by starting with an end-to-end compiler, however incomplete. If I hadn't taken that step, and tried first to plan the perfect IR on paper, I am certain I would have reached an inferior result.
edit: and even the IR and compiler middleware is loosely coupled. The IR is essentially a set of flat data tables, each of which is built independently by walking the AST. And the compiler is implemented in a series of independent passes: i.e. one pass to build the IR, one pass to derive type information etc. so it's very much grown into a series of independent components, each of which could be independently rewritten without affecting the others very much.
A bit of thinking about design and architecture can save you a lot of time. Start with the wrong data structures and maybe you'll have to patch a lot of thing or just redesign everything from scratch.
Be an architecture astronaut and you may never release whatever you're suppose to develop.
It's all about trade-offs.
100% One of my favorite techniques is "Super Pseudo Code" ! Why SUPER ?
Lol cause the "pseudo code" I write can barley be called "code at all" - It is usually just a text-file with a bunch of loosey-goosey-function-calls and parameters.
You know just to get a "feel" for how different entities(classes,struct,tables or libs - pick your poison) will interact and what might be needed. We not talking any UML-Diagrams here - really just text-files and functions/entities
This also works super-well for any multi-step-processes.
The more code your design decision will affect, the more up-front planning it deserves.
You can't loosely couple around questions like these most of the time, at last not without excessive abstraction.
For "how do I structure this reasonably isolated 0-1kLOC component", I agree, easy to fix later if needed.
And like for example if you want to change web frameworks, that's something you can do incrementally. If you're talking about front-end, just find a way to encapsulate your old code behind a clean interface and start implementing new components in the new framework. If you're talking about back-end, then you can just implement new endpoints in a new language if you want even, and gradually migrate things over when you have to modify them.
and also the reuse of the term architecture.. don't put beams at random and see if it works, plan in advance and calculate. They probably had these realizations thousands of years ago, it's a generic economy principle.
I forgot what the saying is but solution unfold themselves when you thought about the problem long enough.
Oh and lastly, Grothendieck said he wasn't in the business of solving problems, but expressing them.
I sort of agree with this, but not quite. I care about code quality, but not code aesthetics. Aesthetics was really important to me when I was younger and more inexperienced. I wanted my code to be pretty, and wouldn't be happy until I found an algorithm or abstraction that met my definitions of pretty code. I would polish it until it was shiny. I would even align my variable names to be pleasing to the eye.
20 years later, I know there are things that will just never quite look neat. FizzBuzz is a pretty accessible example of this phenomenon, but most non-trivial algorithms have that to some extent.
> Besides that, I'm a huge fan of disposable code. In my experience 99% of the time, the best approach is to just dive into the problem and try to solve the problem in the most pragmatic way. Maybe that's hard-coding a lot of things. Maybe that's having a small amount of repetition here and there. Maybe it's writing one long function with 1000 lines.
> After that it's all about continuous improvement. You spend your time solving the problem, and when your code becomes hard to work on, you spend time improving your code to make it easier to work on by cleaning things up, and adding sensible abstractions which solve the problems you actually have.
I too am firmly in the camp of write code first, refactor into a maintainable solution when it works. Maybe it's tacit experience refactoring messy code, sort of knowing in the back of your head how it will shape up when refactored.
But communicating those ideas to less experienced developers is where the problem comes in and all the prescriptive dogma arises.
But this happens in all fields. How many times do experts give some advice they don’t always follow themselves? Experts understand when and how to break the rules, and that comes with experience.
Novices need simple rules to follow. The world of a novice is filled with uncertainty, they have nearly zero intuition as to what's good or bad, so simple rules that get them 80% there are essential. Otherwise they'd get lost in the complexity.
However, with time, as they gather experience, learn, and mature, they should be able to figure out the reasoning behind the rules they were once given. That then will allow them to make decisions on whether the rule is appropriate for the given context or not.
Of course, getting to that point requires continuous improvement, which is why many, if not most, programmers don't get there.
Example: "Never use `goto` in C." - great advice for a beginner, they'd just make their code into spaghetti. However, a seasoned coder knows that `goto` is only a problem when used to jump to arbitrary points in the code, so using it to reduce duplicated error handling is perfectly fine.
I think a lot of it is just lack of context. A lot of "best practice" advice is valuable in certain cases but it gets over-applied, and people end up wasting a lot of time on things which don't matter.
My approach is slightly different.
I think for a while and then start on the path that gets me to what I think is a common case. If the whole seems more complicated than reasonable, I back out the part that seems to be troublesome.
Eventually, I get one case "working". That case might not be complete, but it does something.
I then work on an adjacent case, using the same "back out the troublesome" process, going into a bit more detail (or perhaps deciding that some detail on the first case was wrong).
I repeat "work on adjacent cases" until most/all cases are handled. As I do more cases, I use opportunities to reduce the cognitive footprint.
Yea I understand 100% where you coming from ! I also these days try to focus more on "just getting it done/shipped". Lol I even went so far as to have my own little PHP-MicroFramework called "JGID" which means "Just-Get-It-Done" to remind me at EVERY LEVEL even code that is PERFECT but not shipped is not as good as medicore-core that is shipped ! YMMV
>I think the things I could point to in my coding practice which would make the code I write now better than the code I wrote 10 years ago would be:
I like this approach - instead of asking what is "Good Code" You asking, "How has my code improve in 10 years" a subtle change but I there is a lot of value in there.
This type of introspection-question reminds me of the very very common life-purpose question:
a) What will you do if you have a million dollars ? Sure good question... But I always found the better version of this is:
b)What will you do if you KNOW you won't fail ?
To be honest I think it might have originated from Tim Ferris. You like or dislike him, but what I love about him is his great ability to ask interesting questions.
>- I go for abstractions later, only when I need them, rather than trying to think of the perfect abstraction/design pattern which will solve the problem in the most elegant or clever way
Yup I've noticed this as well. I remember my very first paid coding job. Writing a roulette-wheel-number-reader. And I was so proud to have like a OOP-Hierachy of 3 levels and implementing AbstractFactories for multiple reader-types etc :D Only the "other cases/readers" never happen :)
>..After that it's all about continuous improvement..
Yes - I think I've also seen a similar effect, which is "better code" is a function of repeated-iteration of some process. It's like how woman like men that have a plan. Doesn't even have to be a good plan, just have a damn plan :D
It's funny, but those points you are describing are measures of code quality to me. The less interdependencies the higher I value the code, same for abstractions, most of the time it is the simpler the better. Except in those cases where they are explicitly required.
there's a lot of artistry, poetry, idealism in programming, but as a daily money making activity it just doesn't cut it. You have to reach goals as easily as possible (dry,yagni) fast without painting yourself in corners as much as you can (low coupling, min interdependencies). Finding the right variability points.
it's something school don't do well (saying this humbly) yet crucial
ps: I disagree with the article title. k&r, mccarthy's lisp, prolog, and many other have very clean general and beautiful code.
I only "lead" one project (a minuscule thing by all means) and it was the first time I was responsible, and instead of burning out in rabbitholes, I discovered the bliss of being sharp cause I had near no time to allocate to anything. Suddenly only the minimum amount of change was important and you stop caring for so many parameters. It leads to a saner sense of skills and progress.
You also start to understand systemic ideas better, every patch or change will have your brain think "is this gonna rot fast ? or will it be nicely decoupled from the rest and more pros than cons in the future?". No effort is wasted, every step improves the system. It feels very good, and makes you want to do more of that. Unlike 99% of side projects where you spend days on DESIGN.md or TODO or watching conferences about <trick>
There are many categories of code problem that you don’t have to get right the first time. Some things can grow organically. But there are combinations that can be extremely resistant to change.
In particular, shared state is both state and coupling. Global shared state is a fucking nightmare, and almost made me quit my current job a couple months in.
I thought I could fix it, but I underestimated the number of hands in the code, and the longevity of the alliances I could make to get it done. The thing with a hideous mess is that the people who don’t mind it stick around, and they probably made it in the first place. Mostly what’s kept me here is the pandemic, and he fact that a lot of my energy is now in a hobby, not invested in my job. The tiebreaker was that some self-aware wolves have found slightly better code patterns and I have been exploiting that all I can. Write more like that, delete code like this ASAP. We have spent much of the last 18 months working on things I identified in my first six weeks. Not always in the most healthy way, but at least some people are looking at it.
It worries me that with the leetcode fad in vogue people will pay even less attention to the things which actually matter in real software development.
I think the things I could point to in my coding practice which would make the code I write now better than the code I wrote 10 years ago would be:
- I minimize interdependencies (changing a line of code should not affect something un-related)
- I go for abstractions later, only when I need them, rather than trying to think of the perfect abstraction/design pattern which will solve the problem in the most elegant or clever way
Besides that, I'm a huge fan of disposable code. In my experience 99% of the time, the best approach is to just dive into the problem and try to solve the problem in the most pragmatic way. Maybe that's hard-coding a lot of things. Maybe that's having a small amount of repetition here and there. Maybe it's writing one long function with 1000 lines.
After that it's all about continuous improvement. You spend your time solving the problem, and when your code becomes hard to work on, you spend time improving your code to make it easier to work on by cleaning things up, and adding sensible abstractions which solve the problems you actually have.
In my experience, over time this approach leaves you with a very ergonomic and easy to understand codebase.