I've always wondered why more people didn't actually issue DDL for these kinds of databases. Like if you want to let your user define their own custom fields, just take what they enter and actually CREATE TABLE / ALTER TABLE to build the the tables they need. And then more or less you could just write regular queries.
Turns out sysadmins get REALLY nervous when you tell them your application needs permission to use DDL at runtime. It required a lot more filling out forms.
I was talking about an application and its accompanying database, which would still be paid for and administered by the customer, not a database outside the application which it would be accessing.
That still raised many eyebrows.
If this is the main concern, you could have two databases (or schemas, if your database supports it).
One for application configuration/storage that the application never runs DDL on and another for user data, where the tables can be created/modified dynamically.
That separation means that, at worst, user data is screwed up if there's a bug in the DDL generation, but the application should still be able to run.
At least, that was my read.
A bunch of dictionaries containing UUID/data, where each dimension is its own dict, and the dicts might or might not be running on the same system?
Well, technically, the database's own system tables have tables with such column names and responsibilities.
Btw I think your SQL example is called "EAV Schema" and does have a legitimate purpose once in a while.