Datomic already sort of does this :) You configure a storage backend (Datomic does not write to disk directly) which can be dynamodb, riak, or any JDBC database including postgres. You won't get readable data in PG though, as Datomic stores opaque compressed chunks in a key/value structure. The chunks are adressable via the small handful of built-in indexes that Datomic provides for querying, and the indexes are covering, i.e. data is duplicated for each index.
Interesting! I assumed Datomic was entirely custom
Now I'm even more curious if you could skip Datomic and just do something like this directly with a relational DB in production
Yes, but you end up rewriting Datomic!
Well, except it sounds like Datomic is closed-source :)
Nah, our (not very good) implementation existed 10 years before Datomic :-)
But why?! The whole point of Datomic is that it implements this entire immutable framework for you, on top of mutable storage.
So YOU can focus on building your own specific business logic, instead of re-implementing the immutable DB wheel.
Because, for example, your application is not tied to the JVM? You are uncomfortable using closed source software for such a critical piece of infra? As far as I can tell they don't even have a searchable bug report database! I'd hate to be the one debugging an issue involving datomic.
You could have one or more append-only tables that store events/transactions/whatever you want to call them, and then materialized-views (or whatever) which gather that history into a "current state" of "entities", as needed
If eventual-consistency is acceptable, it seems like you could aggressively cache and/or distribute reads. Maybe you could even do clever stuff like recomputing state only from the last event you had, instead of from scratch every time
How bad of an idea is this?