However...
There is something sort of new from the academic side that could potentially be a theory that can deal with problem you describe.
At a high level the theory looks almost like a formal language for "design" and "abstraction." It's called category theory. It's new-ish so it doesn't yet solve the problem you describe but I see the outlines of it within the framework of the theory.
In terms of just pure math though, category theory is very new when compared to most of the disciplines around today.
Also you are completely wrong about it being a century old. It was invented in the 1940s.
I'm assuming you just googled it to find counter points. That being said you missed something even OLDER then CQL and the most popular and definitive use of CT in programming today:
Haskell.
Anyway my main point is that Category Theory can answer some questions about 'deriving' an architecture for a problem rather than having a person 'design' one. The way category theory is used and haskell and CQL is as a framework. In haskell and CQL it's more about designing the system within the constraints of category theory then it is about deriving a system. The derivation of a system from a formal theory was ultimately what I was trying to get at.
You can sort of see an outline for the possibility of this happening when you study CT. I'm being really speculative here in case I failed to convey that. Such a system doesn't actually exist (yet). That's all.
Category theory can directly give us systems from theories. See the papers behind FQL/CQL for more details, especially [2] and [3].
[0] https://wiki.haskell.org/Hask
[1] http://www.cse.chalmers.se/~nad/publications/danielsson-et-a...
Is it just that it's not popular or are there flaws?
Since category theory looks almost like a formal theory of "design" and "abstraction" is looks like if one day someone could come up with an algorithm or function to formally derive a solution given a set of requirements. To anticipate changing requirements you simply need to adjust the initial set of requirements to be more broader and then this hypothetical function would output a system that can handle potential changes that you anticipated.
This is all hand-wavy and not concrete. Many people who study CT see it as currently the closest thing to a formal theory of "design" and "abstraction." If this is true then techniques or algorithms for deriving architectures of systems can become turn key.
Of course let me emphasize, I'm just speculating here. Not talking about anything that actually exists. Just saying that if you study CT you can kind of see this kind of thing existing sometime in the future.
Hoar logic can prove your programs 100% correct. This is useful for anyone who wants to write a program that's bug free.
UML is more of a visualization tool then a theoretical tool.
>Proving diagrams commute with abstract nonsense is not going to help the average programmer.
What in the world is a proving diagram, and what is abstract nonsense? Please be more clear with your explanations. I am not talking about diagrams and trying to say they commute with anything.
That's true, only if you can assume perfect knowledge about the problems/requirements, the data, the environment, and how (if at all) those will change.
In practice, none of these tend to be true, and that is where a lot of gut feeling will come in.
An experienced practitioner will just know that certain modules / feature sets will grow and grow and grow over time, and will take special care to build a particular robust approach for that area. Or that certain kinds of data are often of low quality.