Lines like this are, at best, a gross oversimplification of relational database performance tuning: "But when you need to optimize performance, you look at the queries you actually perform, then merge tables to create longer rows, and do away with joins wherever possible."
And this: "We require sub-second responses to queries."
Really? Single big-box OLTP systems have many issues, but if you are getting query times greater than a second on a typical online database then you are either doing something wrong or have specific requirements. Neither of those things will be fixed by blindly picking 'NoSQL' over 'SQL'.
"any significant database needs to be distributed." I could easily list, off the top of my head, 100 significant databases that aren't distributed. Maybe some of them should be, but saying that "any significant database needs to be distributed" is a real stretch.
> There are many good reasons to design around a NoSQL database instead of a relational one. This article provides fewer than zero of them.
Yeah, that's the thing. An article of a quarter of the length with more research, less strawman bashing and less breathlessness could have made the case for NoSQL much more effectively.
This one gave me a laugh. Thanks for pointing it out.
Let me see.... Your business does nothing but manage money. You don't need absolute confidence over where that money is at any given point of time? Right..... In fact accounting systems (and by extension ERP systems) are about the LAST place you'd want to use anything other than a relational database system/
At a low level banks use both absolute consistency (for physical transaction stores) and eventual consistency (for logical transaction implementations). The logical transaction implementation abstracts inter-bank and physical payment messaging.
It would be impossible to have absolute consistency in the logical transaction layer as transaction scopes would have to be open (i.e. locked) for days at a time in some cases. That simply doesn't scale.
Ultimately banks have millions if not billions of pounds floating around not in traditional transactional stores all the time.
But loose coupling between third party payment networks (say debit card purchases over Cirrus) and the bank is not really the same problem as using Cassandra at Facebook.
There's a paragraph in the (long) article about absolutely local / eventually global consistency.
>"but they are hard to partition without extreme effort or extreme cost." ... Wh... Bu... That's not even wrong.
I don't know about Tandem but another comment mentioned Postgres-XC so I checked it out. According to the wiki its not globally consistent.[1] Care to elaborate on what isn't wrong, or name any (relational) alternatives to Postgres-XC?
"We need some research work for solutions on following issues.: Global constraint. Can we enforce unique or other constraint exclusion globally in multiple data nodes?" http://wiki.postgresql.org/wiki/Postgres-XC
Maybe I'm thinking too close to the metal; to get eventual consistency at a higher level, you still need atomic consistency at a lower level - even Paxos uses two-phase commit under the covers.
As for the partitioning, what "isn't wrong" is:
- He says the CAP theorem is historically irrelevant to relational databases, which is wrong, and that they have been "adapted" to provide high availability, which is kinda wrong, and then:
- He confuses "partition" in the "partition my database" sense (scale my database across multiple tables or nodes) with "partition" in the original CAP sense (do not permanently explode if a node goes offline), so he isn't even wrong - any multi-server database allows partitioning, by definition.
Also, I think you're misunderstanding that item on the wiki page; it sounds like Postgres-XC IS globally consistent, but that it can't yet support some important forms of consistency like UNIQUE constraints across nodes (you can only have a unique constraint within the same node). All nodes would see the duplicate rows, though, so it is globally consistent.
- I have never seen changing schemas on the fly in a RDBMS programmatically (except admin functions). I have seen overloading of column types however.
- "super column" data stores have a deistinct design advantage that you can not get from relational DBS.
- everyone that is using relational does the same thing when they get big data: denormalize. From materilzed views al the way to sharding and replication.
- banks use transactions, but not as a two phase commit. it is a single atomic record - not partial updates.
- we still have some species from the age of the dinosaurs, but the land is rules by those that have evolved.
- I agree that banks should be consistent. Not sure what he author was saying here
- CAP is relevant to ALL datastores. Consistancy, Availability, Partition tolerance : pick any two.
- if your post does not get committed to Facebook, it may be important to YOU, but the application is designed more for availability
I'm not sure what you think two-phase commits are, but they ARE atomic transactions. They are essentially "Everyone cool with this transaction?" "Yep!" "GO."
The CAP gotcha is that you already have partition tolerance - if you didn't, your database would go "boom" when a node went offline. So, really, it's "pick one".
And believe it or not, Facebook moved some stuff from Cassandra to HBase in part because of its stronger consistency model.
Two-phased commits are when you write a partial record to the DB and then cleanup the transaction. When they complete succesfully, they are a complete transaction, but during a failure they are not atomic through the database because it takes extra application logic to cleanup.
Partition tolerance is not when one node goes offline, it is when critical nodes (master) or up to n/2 fail.
For CAP, you can optimize for just Consistancy, Availabilty, or Partition tolerance. But you can also pick two - you just can not do all three.
I did not know Facebook went to Hbase and why it do so - can you provide a link?
"The network will be allowed to lose arbitrarily many messages sent from one node to another."
Which means that, if you have a network, you have partition tolerance, period.
But Stonebraker defines it differently:
"If there is a network failure that splits the processing nodes into two groups that cannot talk to each other, then the goal would be to allow processing to continue in both subgroups."
Great article here:
http://www.cloudera.com/blog/2010/04/cap-confusion-problems-...
As for two-phase commits, I'm not sure what system you've used that requires app knowledge, but (for instance) PostgreSQL does them automagically; you just do PREPARE TRANSACTION and COMMIT PREPARED, and if the transaction fails you need app-level error handling - but presumably you have that error handling even without two-phase commits.
Here's an article on Facebook's move to HBase:
- Relational databases were designed for a world where availability is unimportant, like transaction processing. Um, no. OLTP has four letters, not two, and the first are as important as the last. Tandem was providing five-9's systems in the '80s and '90s for ATM networks, lottery systems, airline reservations, credit cards, etc. Tandem is a fault-tolerant relational database in hardware.
- Up-front schema design is a poor fit in a world where data requirements are fluid: I don't think "up front" means what you think it means. You can change schemas on the fly nowadays; your schema design is no more up-front than your coding is.
- You can't have millions of columns in a relational database: True, and you wouldn't; you'd normalize that. This is an important difference, but not a disadvantage of relational databases, any more than saying "In a relational database, you'd join URLs with IP addresses, and maybe five other tables; this design isn't even conceivable in a NoSQL database."
- To optimize relational performance, you "do away with joins wherever possible": 1995 called, and it wants its MyISAM back.
- Two-phase commit is so obsolete, even banks don't use it: Of course they do. You still need a two-phase commit to make sure the other end got your data; whether "got your data" happens in the customer path or during reconciliation is a design decision. How, exactly, do you think they discover that you and your spouse both got the money?
- "relational databases were developed when distributed systems were rare and exotic at best." That's nothing; when Von Neumann machines were developed, we didn't even have transistors. Some legacy architectures keep on working.
- "absolute consistency isn't a hard requirement for banks": see above. Yes it is.
- "So the CAP theorem is historically irrelevant to relational databases: they're good at providing consistency, and they have been adapted to provide high availability with some success, but they are hard to partition without extreme effort or extreme cost." ... Wh... Bu... That's not even wrong.
- "consistency requirements of many social applications are very soft." I like the Facebook example from a recent article on causal consistency: I de-friend my boss and then post that I'm quitting. Certain kinds of consistency are in fact critical to social applications.
There are many good reasons to design around a NoSQL database instead of a relational one. This article provides fewer than zero of them.