I think that generally I'm maybe a 5x programmer when it comes to architecture. Architecture is what I'm good at.
But when I come onto a team for the first time, they don't just give me the keys to the kingdom. I don't get to make large-scale architectural decisions immediately. Instead, they have me work on smaller sections of the application. And when I'm working on smaller sections, the attributes that make me good at architecture work against me. I generally work slowly, taking a step back to understand the problem from the user's perspective, and modeling that understanding in code rather than writing code that short-sightedly solves only the problem at hand. That works well for architecture, but in smaller sections it just appears slow, like I'm not producing. It does have positive effects in the long term even for small sections of code, but those benefits aren't seen until long enough later that I don't really get credit for them. The result is that I don't make a good impression at first and it takes me a long time at a new job before I'm given authority to make architectural decisions.
It would be easy for me to blame other people for not seeing my "genius" here but the truth is that this is only a problem I can work on--I need the flexibility to be able to demonstrate productivity early on and make a good impression, otherwise my poor self-presentation gets in the way of me contributing to the team in the most effective way possible.
So I guess what I'm saying here is, you need both. It's not enough to just have elegant architecture, you also have to have quick programming skills.
Quite often, simpler and less code makes it easier for others to participate.
Joining an existing project requires a demonstrated ability to build the trust to refactor and re-architect anything over time. Otherwise patchwork works.
Where good people win is that they adapt quickly when there are signs that the design doesn't work. A lot of people just "stick to the plan" although everybody knows that it won't work.
Instead, you should look at how those requirements will be produced. If your architecture "thinks" the same way as your users, it will be able to be modified easily to include more of their thinking. So instead of trying to predict future requirements, you should attempt to capture and accurately model the way your users think about the problem. Sometimes that creates designs that mathematically/algorithmically seem pretty ugly. But in the long run I'll often discover that the ugliness exists for a reason--people mostly only add complexity to their mental models because that complexity allows them to solve a problem. So even when the domain expert's thinking seems overcomplicated, it's usually actually the least complicated model that accurately represents the problem.
Rather, listen to your users problems, listen to what they think a solution is, then let your design team determine if there is a better and more simple solution. Sometimes there may not be, but often there is.
When you get requirements from your users, they're going to mix in how they think about the problem with how they think that problem should be represented in the program. The problem with that is that they don't know what computers can do, so they can't possibly be expected to know how the problem should be represented in the program. And in fact, the UI, which is the only thing they ever see, will probably be more task-oriented than representation-oriented, so they probably will never see how your program actually models the problem and therefore won't be able to suggest good changes to it.
Ultimately, though, there's an underlying mental model that the user is using for the task. And for most fields, there's really only one correct mental model, so I actually disagree that expert users have different mental models, at least not with significant differences.
Quite often in super early projects, the domain is not understood, so the architecture reflects the learning (or misguided assumptions).
It's not about that, it's about keeping what you do nimble enough to always be ready for change when it comes. That means not having duplication in the system, having things well factored and encapsulated so there's a distinction between the abstract and the implementation so at any time any component can be made a change point and become pluggable. What makes a good architecture good is that it can be changed without scrapping the whole thing or breaking the whole thing.
> Where good people win is that they adapt quickly when there are signs that the design doesn't work.
It's not that they do adapt quickly, it's that they "can".
> A lot of people just "stick to the plan" although everybody knows that it won't work.
Because they're stuck with a bad design that can't be changed due to issues like bad factoring, lots of code duplication and copy pasting, etc.
From my experience they just don't try. There are (almost) always ways to improve.
I have a few enterprise projects now over 10 years old that began as simple projects. It took having enough understanding of the domain and problem set and then anticipating what was expected to grow and change, and what did not.
No design is complete in the start. It needs to find a balance between being a foundation for exploring and developing through the problem domain, but remaining flexible. I think a lot about managing and mitigating risks through clever and simply designed architecture that can do a lot. The less there is to the design the more it can remain flexible to deal with new facts.
The fine balance of over architecting/abstracting vs under abstracting/architecting is for me a matter of finding the balance between the size, impact and possible number of unknowns about what I'm trying to design when I begin vs the future. Where possible, understanding the domain before daring to design or interpret is often the most important factor.
For the few large enterprise systems I've designed in unrelated fields in the past 10 years, I haven't ended up throwing away or majorly refactoring the schema or design. What did change is the stack and tools that became available and making sure I kept a laser focus on an n-tier architecture so different layers of the presentation, business logic and app logic could be swapped in and out over it's lifetime. I guess I can say comfortably that risk of designs falling apart was minimized, and luck has increased because I didn't begin without an understanding of the core principles, entities, and interactions ultimately the details and workflow the system has to manage.
I can't stress the importance of keeping things as simple as possible, because complexities are guaranteed to arise in the future.
To me, features, and schema changes over time can become like perpetually flying bullets.
I absolutely have refactored or adjusted as I go? Sure. Does all of it get thrown away? Rarely. Have I over-architected a solution to cover my butt and the new facts never came, totally. Has a new tech, stack or library come out that made me need to replace what I had built or modified to do what I need, always.
I do lots of paper models to try and see the core patterns, and what could be interpreted in a number of different ways should something grow in that direction. This type of strategy is far more difficult when working in a problem space where there is a lot of research and development.
How about you? Would love to hear how you approach things that have different balances of known/unknown facts.
And clever elegant architecture and clever programming will beat either alone. It's not an either or situation, you want them all.
Clever and elegant architecture will always beat clever programming.
It's almost always a large part of how "10x programmers" achieve so much.
10x effectiveness comes out in different way on different projects, be it new code vs existing code, or knowing enough about the solution space vs being in an unknown world.