Naively one would expect that no individual UPDATE or INSERT statement could deadlock in isolation… but there you go.
If that is possible, the possibilities across multiple concurrent data mutating queries are beyond human comprehension!
Serializable should absolutely be the default!
Similarly, all columns should be automatically indexed to at least some degree, like Postgres BRIN indexes at a minimum.
Time and experience have shown that the vast majority of developers are pathologically unable to properly define all required indexes ahead of time.
That being said also "required indexes ahead of time" is impossible - because query patterns change and things get released unless you mean something like "the required indexes for our obvious query patterns we just freaking released.
I dont think most columns need to be indexed because that is mostly nonsense, most columns never have a where clause applied to them or are sorted on themselves, so an index provides ??? value.
As a consultant I come across a lot of CotS software, in-house or otherwise bespoke software, etc. Roughly 40% of the former has 100% of the minimum required number of indexes and approximately 5% of the latter. By "minimum", I mean the indexes required to avoid full scans of tables that will become large enough for this to be a problem in production.
"Disciple doesn't scale." is one of my favourite sayings now, for a reason!
1) Developers almost always work with toy data, and are hence insulated from poor indexing decisions. Problems turn up 'x' years from now. It is well established that humans learn poorly when consequences are delayed... by mere hours, let alone years!
2) DBAs and developers often have an adversarial relationship. A common consequence of this workplace dynamic is that developers aren't granted the required access to tune indexes, especially in production, which is where the issues manifest.
3) I've heard anecdotes, including here, along the lines of "XYZ cloud native / webscale database is so much faster than ABC traditional RDBMS!". Very often the difference is just that XYZ auto-indexes by default. CosmosDB, Google Firestore, Kusto, Elastic, Druid, and many columnar formats are in this category of "magically" faster!
I'm now 99% convinced that RDBMS needs to be reinvented for the modern fast-paced, vibe-coded, "I'm a fullstack(lol) dev" world where people simply don't have the bandwidth to pay attention to minutiae like on-disk sort order and filtered secondary indexes. A better fit for today's world would be a system that is: columnar by default like SAP HANA, compressed[1] by default, indexed by default (thanks to being columnar!), serializable by default, and "include batteries" like native queue capabilities so that nobody has to figure out cross-RDBMS complications like distributed transactions, outbox patterns, or deal with the consequences of a DBA rolling back one of two databases to a backup.
You missed adding the reference to [1], I'm curious now.
AI buzz aside, unified multimodal data platform is an interesting idea.
Disclosure: I work part time in the DB group.
MS SQL?
We've had to set MAXDOP 1 on some specific queries for a long time.
but ignoring that, serializable isolation level means the database acts AS IF the transactions are serial. but most databases in fact will execute them concurently, with careful tracking to make sure they appear serial
Sometimes though, yes definitely. It's hard to claim anything universal at all about databases.
(Unless you mean "being able to choose between different isolation levels", then yes, completely agreed. Very very few use anything but the default, somewhere below serializable, and it always concerns me unless they can describe exactly what they're intentionally allowing and why it's okay for their system. Most cannot.)
But you usually don't need serializable for this, cause READ COMMITTED locks the rows during updates.
Big point: Serializable not exist alone in a decent ACID datastore, and no, less strict rules for the MOST important thing you have(your data) is NOT a good idea.
Over and over again Acid RDBMS have proven that trying to "relax" the rules in pursuit of performance or worse, mystical holy grails that have never been right or correct for a primary datastore, is a mistake. And then people goes back to them, because is the best tool for ALL the primary data store jobs. ALL OF THEM.
Is like the mythical C developer that "not need safety", that at least has more chance of be possible (after MANY passes over the code) that a datastore without safeguards.
On the contrary. The most successful RDBMS by far was MySQL in an era where it didn't have any kind of ACID (you could write the transaction keywords but they didn't do anything). As the story we're talking about now shows, RDBMSes are routinely deployed with transaction settings that their users don't understand, much less use; there are settings that would reduce bugs if anyone cared to use them, but no-one does. People cargo-cult the idea that they should be using an Acid RDBMS but they almost never actually want or need one.
Pretending like redis is suitable for an RDMS workload because it executes things serially means you’re completely ignoring what transactions are actually used for and how they work.
On the contrary, you have Lua which is much more expressive than SQL (yes Turing completeness, but there's a huge difference in how easy it is to read and understand).
> Pretending like redis is suitable for an RDMS workload because it executes things serially means you’re completely ignoring what transactions are actually used for and how they work.
Well the vast majority of RDMS workloads don't use serial isolation, that's part of my point. As for the rest, all I can say is I've worked on many systems in many industries and seen very few (honestly none) that actually made effective use of what transactions do and don't give you.
SSI gives you the performance of close to snapshot isolation with the safety of serializability. The lowest I would ever go is snapshot isolation and ideally SSI. Anything else I’d use an eventually consistent database instead because then there’s at least no pretense.
Sure, sometimes it's too slow, but it should be the default.
Very few people can write correct database code at the other serialization levels. Most think they can, but it's harder than correct multi-threading, because databases do weird unintuitive things for performance.