I think this is a major benefit of design docs - they are a way to extend your engineering influence beyond your own individual output. If you write a design, and your design allows you and three other engineers to coordinate your efforts, then your engineering output is now "I coordinated a team to build something that any of us couldn't have written individually."
This dovetails nicely with your next point - uncovering blockers as early as possible is critical when coordinating a bunch of entities. Project plans are usually written on the assumption that every task will succeed, but there may be extra tasks added. If a task cannot be completed / you need to redesign something / etc, this will suddenly bring the work of N engineers to a halt. The earlier you successfully split the work into a series of known unknowns and implementation tasks, the better the project will go in general.
I think the challenge for me has always been that the "uncover blockers" piece means building one or more small prototypes to prove out the capabilities of the dependencies, check feasibility, etc. So the building of these prototypes occurs prior to or in parallel with the authoring of the design doc, but then at a certain point they get paused so that the design doc can be completed and reviewed, and then picked up again when it's time for the "real" implementation to occur.
But pausing there takes discipline, since it ideally happens at the exact moment when all the main blockers have been cleared away and it is maximally tempting to just step on the gas and start into the work of cobbling the prototypes together into the project.
That analogy falls apart quickly. Design docs aren’t specific enough to be analogous to blueprints.
You can give a set of blueprints to 3 different construction firms and get fundamentally the same building. Try giving design docs to 3 different development shops and see what happens.
The problem is that the only way to get to that level of specificity is with code.
Design docs are closer to something a city planner would produce than to something an architect or civil engineer would, and they should be treated accordingly.
Different jobs require different levels of abstraction and design docs fulfil that role for software engineers.
A bad analogy is like a leaky screwdriver.
Nothing could be more incorrect when it comes to software. In software, the design is the code. The compiler/interpreter are the construction of the system.
That's IMO a pedantic view. Yes, ok, but when I design software I describe it's high-level characteristics, more like describing the plot of a book than the specific words in a book and the what the characters say to each other.
I'd say there are many levels of design. Code is the last.
Each stakeholder has a different "design" in mind, and until you actually get specific there is no design, there's just a nebulous, incomplete list of requirements. And if you do try to get specific enough to be reproducible--you're writing code.
None of this is to say that design docs are worthless, just that they can never be specific enough to function as the actual reproducible design the way blueprints would. Thinking of them that way is harmful.
Construct a software development methodology around that assumption. Now remove that assumption, and think about the practical consequences for a team who doesn't realize that the assumption is incorrect until after they have started implementing a design document.
Imagine what would happen if you hired an architectural firm to draw up some blueprints for a house, and then midway into the construction of the house it turns out that the assumptions made in the blueprints are wildly out of sync with what's possible to build.
It is significantly better for everyone involved if management goes into the process knowing the limitations of design documents and up front design.
For more concrete example of the practical problems see the history of the debate between iterative and big design up front methodologies over the last 30 years.
The right analogy for design artifacts really is to the higher-level vision definition for a project. Engineers don't just show up and start drawing blueprints; one or more people come up with a purpose and concept for a building project, brainstorm approaches, evaluate trade-offs, come to some consensus on the right direction, and only then start working with engineers to start creating and iterating on blueprints. This vision phase is where design docs fit.
My first flat had a toilet that blocked the door from opening fully. It was correct according to the blueprints.
The thing to take away from that idea is that the design doc is not law written in stone, but a base for changes. It keeps your design honest on the high level. If you end up clashing with reality, and your design doesn't work out, you can backtrack to the design doc and refine it as necessary.
In short, the design doc should be a living document.
In software, if the code is the "design" then it should be a priority for computer science to agree upon a standardized design representation (UML) that can be generated by the compiler/interpreter, along with code coverage and static analysis.
The issue here is that there has not been adoption of formal rigor of design and definition of implementation into code of proof of correctness alongside data structures and algorithms appropriate to the domain state. Add in understandable and usable build and testing tools.
The infrastructure is there, with CI/CD, and a repository of github/gitlab, team foundation, cvs code and say something like a successor to GPT3 to do the grunt work of analyzing code to determine optimal patterns of implementation vs existing code bases and development sprint branches.
Even compiling code or "making" it can be a crapshoot for extremely senior and competent developers.
Some food for thought.
If you hand one complex blueprints to three random construction firms, from a distance you’d get the same result, nearly, but up close a lot would’ve changed during the project.
There is a reason why engineers are required to inspect the project at some interval and perform quality checks.
Blueprints for a house are much, much closer to being reproducible than design docs for a software project.
If you give design docs to 3 separate dev shops, the end results will be wildly different.
>There is a reason why engineers are required to inspect the project at some interval and perform quality checks.
It's true that there is still some room for interpretation because blueprints are still a model. But most of what your talking about is because people will cut corners and not follow the specifications, not because the specifications aren't there.
A well written piece of code will only tell you what it does, a good suite of tests will teach you how to use it, but only natural language documentation - be it in comments or separate documents - will explain to you why the code exists in the first place, and why it looks the way it does.
(As for "self-documenting code", unless a lot of your functions contain the word "because" in their name, the code isn't really self-documenting.)
I wish this was more commonly shared and understood
I don't know if this is actually controversial, but I would not want to work at a place where this is controversial.
Apologies for the diatribe. Big fan of design docs, but an even bigger fan of software engineering cultures that focus on simplicity and usability, rather than being everything to everyone (or worse, being a pet project of an exec who changes their mind every quarter about how things should work).
As it stands now, we take high level feature design docs from PM, and turn them into service. Everything between that, resources, development, operations are handled entirely in team where everyones title is SDE. This placed a lot of communication, writing, designing on us but I'm not complaining.
I achieved the outcome, mostly with bad to average code that slowly improved over time.
When I finally understood the domain to make bigger scoped decisions, I started doing design docs beforehand and my code continued to improve greatly.
Now when assigning out work to more junior engineers I find myself giving them a high-level design doc, with some detail missing, they deliver higher quality work than I did at that stage, and they also seem to upskill faster.
This however depends on me making the right decisions at this stage, which is not always the case, so not fool-proof but an overall software quality improvement has definitely occured.
Does anyone have any insights on how to make the reviews more constructive + quicker?
1. Start with defining the problem you want to solve and identify the stakeholders. Then meet each stakeholder 1-1 and get approval.
2. Propose an API. Then meet 1-1 again for approval.
3. Make a high level arch diagram. Seek approval again.
4. Proceed with actually writing the design doc and begin the formal and official review.
Points 1-3 could be done in a week if planned well and will make the design doc a lot less controversial and get you through the review process quicker
- Have a single person (engineer) who is responsible for making final decisions. Gather input from others but there should be one individual who is responsible for making sure that the design has a consistent vision that meets the goal. If a non-critical decision is taking too long, just make a choice and run with it.
- Be clear about the goal of the project. At Amazon they use the PRFAQ process where you need to begin by writing a costumer-centric press release that basically details why customers should be excited about the project. This helps keep design discussions focused on the customer.
- Be clear about the non-goals or things you will not do.
- At the beginning of a review meeting, have everyone read the document. The document should describe all relevant context. At the end of reading the doc, everyone should have enough context to understand and discuss the design - everyone should have the same understanding of the design (i.e. no confusion about which version of the doc we're talking about, no one who didn't have enough time to read the doc beforehand and just skimmed it).
- Have people add comments as they read through it, the authors can respond to the comments in real-time and once everyone is done reading the doc, the comments act as a list of things that need to be discussed.
- Have the doc be the absolute source of truth. If it's not in the doc, we're not planning on doing it. This prevents cases where you have a chat with someone about an idea, look into it further and decide to not do it/do it differently, but the reviewer thinks you are still going to do it, but you just haven't updated the document yet.
I have also been at a larger company, so you bring up a good point about fighting for work. I also think this is right.
1. If you win the fight for high visibility work, that's a good sign that you're the right person for the job. It wouldn't make sense for the company to have you work on something for your own benefit over the company.
2. You may opt not to fight and discover other problems to work on that have strong potential for being high visibility. Maybe that's something relegated to PMs or eng management, where the engineer might feel helpless to the given assignments, but that's more of a culture problem. I think politics increases when there's less work/budget/praise than there are people, which is more situational than cultural, in my opinion.
For old things and complicated things, though, they were how things could be attempted to be built. And with the builder being someone else, there had to be an audit from the build to the blueprint, if you really want confidence in that statement.
If people get promoted in spite of not doing a good job of documentation, now you know why it doesn't happen.
I know this is fairly controversial, but our jobs isn't just to write code. Navigating organisations and achieving consensus between a lot of teams/technologies is a huge part of it.
Design docs are a way to get all of that out of the way _before_ writing thousands of lines of code. The review process nets significantly different feedback to code reviews too.