Unfortunately, at a very deep level, that’s simply not how RDBMS works. The tuples are a B+tree, and in some (MySQL [InnoDB], SQL Server) cases everything is clustered around the PK. If you don’t create a data model that’s easily exploitable for optimizations designed around that data structure, you’re gonna have a bad time. It’s no different than if you decided to use strings to store ints – you _can_, but it’s a bad idea for a variety of reasons.
What you can do is give yourself as much leeway as possible, by following some basic best practices. For example, it’s a hell of a lot easier to update a tiny reference table than to update billions of rows when you decide that column `region` should say `European Union` instead of `EU`.
My dream is to have a tool to model my logical data model and then it will organize my data into the best storage and caches.
I don't think any existing database today is useful.
Schemaless was one of the original drivers for NoSQL databases.
Now, when I need something schemaless, I start with a Postgres table with an ID and a jsonb or json field... which at least makes it easy to have a schema when the inevitable happens and schema-dependent code ends up getting added to the project.
> To do this every data dependency in the system needs to traceable.
This is a hard problem.
I wish we would develop software where the data model could easily change.
To do this every data dependency in the system needs to traceable. Nothing does this so far. And everyone just picks a database off the shelf but none are even remotely useful for this.