Re:Design Considerations for High-Throughput Cloud-Native RD...
by ddorian43 on Wednesday January 16, 2019 @05:23AM
(#18919638)
From a high level things look doable, but going deep in the c/c++, they're not.
Re:Design Considerations for High-Throughput Cloud-Native RD...
by willvarfar on Wednesday January 16, 2019 @07:16AM
(#18920162)
I poke around in mysql storage engine code, among other things, and its entirely doable. It just needs to be a goal. Wish it was on my day job list of things I can work on. I hope someone else has an itch that needs scratching.
The big reason that DDL is slow is because these systems haven't tried to make it fast.
This is, of course, a contary opinion so hear me out before judging me ;)
My thinking is thus:
There are lots of virtual storage engines in the mysql world such as 'federated' and 'spider' and 'union' and such. These actually abstract away more than one data-source, and often the engine is smart enough to support when the data sources don't have identical schema.
These virtual storage engines demonstrate that a layer of abstraction can cope with casting queries across more than one non-identical tables.
So, either built in or as a storage engine abstraction, databases _could_ support DDL changes by putting rows with each version of the schema into actually different tables, and casting queries across them etc transparently.
Another approach is that taken by the postgres engine, where each row has a version and some DDL such as add column with default null can be done instantly. (With a bit more thought, even defaults could have been coped with instantly; its a shame they weren't.)
So, blame the DB designers!