John makes some good technical arguments about why implementing transactional SQL on top of a distributed KV store (even a transactional one) is hard.
The points about metadata performance and consistency were actually new ideas to me. I already had beef with moving the data around to SQL nodes as that is an obvious waste of capacity.
But the importance of metadata in processing SQL queries in a distributed database never occurred to me even though I've lost a decent chunk of my life implementing that consistency. It's one of those things you take for granted if it's how you have always done it.
Seems like the guy knows what he's talking about as well, b/c surprise, he's working on a DB.
https://gigaom.com/2013/03/27/why-apple-ebay-and-walmart-hav...
Teradata is VERY expensive and my guess is reaching also scalability limits.
I'm not sure why Apple would prefer FoundationDB to Cassandra for this usecase.
Now I don't think FDB (the product) is the answer, at least not in the short term. There are more problems scaling it to Apple's use case than there are working around Cassandra's lack of ACID.
So I'm convinced the value of FDB is the experience in the engineers' brains. Apple need brains to run Cassandra, but also to figure out if Cassandra is the right long term path. Build, buy, adapt? It takes veterans to make the right call.
My guess is that FoundationDB is replacing their Teradata installation. Better to buy the company and invest heavily in it then let it not met it's full potential as a small startup.
A more measured, intelligent consideration of pros/cons is needed.
Every April 1st :)
The only thing I can think of is Apple has an internal team that is building a Cassandra replacement and they need more qualified engineers with experience.
Yes, it's critical about the "layers" model and about the SQL implementation in particular. Most of what I write isn't this critical, but I thought there was actually an interesting point here so I wrote it down. Take it for whatever it's worth.
My primary issue is that the people best able to rebut John's well-argued points are no longer able to do so.
My secondary issue is that John could have easily made the same logical arguments _before_ FoundationDB was acquired, but chose not to do so for whatever reason. This would have led to a much more enlightening debate than the one we're able to have now.
It's actually precisely because FDB isn't a perceived competitor anymore that I can write this. As a vendor, I actually have much less of an agenda now. It's not like I'm worried about FDB stealing a customer. If I had posted this months ago, it would have been less credible and it would have felt tacky.
The other point is that I think there's actually something to say here. I'm not just dumping on a dead product, I'm trying to show there's a lesson to learn here about trying to bolt SQL onto things (a trend). To contrast, if NuoDB disappeared tomorrow, I could write a solid post on why they could never technically achieve what their marketing said they could, but there's no lesson there. "Don't make bad engineering choices" is too generic.