"Together, the Record Layer and FoundationDB form the backbone of Apple's CloudKit. We wrote a paper describing how we built the Record Layer to run at massive scale and how CloudKit uses it."
I think this is the first time that little detail has been publicly disclosed.
> your SQL database
They mention "a declarative query API", but as far as I can tell that's not actually SQL, right? So migrating from another relational db would require learning a new query language?
The client/server distinction isn't terribly strong in the FDB world. The FDB client is unusual in that it's a (stateless) part of the FDB cluster itself. You could therefore embed it in the client itself or build an RPC service around it. The Record Layer takes the same approach---it's just a Java library---so you could either embed it in the client application or build some kind of wire protocol for accessing it. One could have an embedded SQL layer like SQLite or H2 with no additional server beyond the cluster or a separate SQL layer network server that acted more like Postgres or MySQL.
The Record Layer was designed for use cases that don't need a SQL interface, so we focused on building the layer itself. That said, the Record Layer exposes a ton of extension points so there's a fluid boundary between what needs to live in its main codebase and what can be implemented on top. There are almost certainly enough extension points to implement a SQL interface as another layer on top of the Record Layer. For example, you could add totally new types of indexes outside of the Record Layer's codebase, if that were needed for SQL support. It's still a lot of work, especially on the query optimizer. Perhaps the community is up to that challenge. :-)
"In the future it is possible that the Record Layer may develop a formal query language, but it is unlikely that such a language would closely resemble the SQL standard." [0]
[0] https://foundationdb.github.io/fdb-record-layer/FAQ.html
* Full table scans on large tables in a R/W transaction would fail due to the 5s transaction duration restriction. This is obviously a bad idea regardless of database, but if you wanted to support SQL you would have to allow it
* Sorting and joins require (theoretically) the same amount of memory as your data size, or the ability to spill to disk. This isn't FDB specific, but offering this kind of feature in a scale-out, high-concurrency model would be tough for QoS.
There are plenty more, but those are the big ones I ran across during design. The actual SQL interface part is not hard, but you would have to disallow many useful constructs people normally expect to work.
So far behind it that they already shut it down once.
FoundationDB excites a lot of people because it's an extremely scalable and extremely reliable distributed database that supports ACID transactions, and which is both open-source and has Apple standing behind it. And yeah, all of that is pretty nice.
But arguably the real power comes from the fact that it exposes a relatively low-level data model that can then be wrapped in one or more stateless "layers". All of these layers write to the same storage substrate, so you can have your document database, your SQL database, your time-series database, your consensus/coordination store, your distributed task queue, etc., etc., but you're only actually operating one stateful system. Your SREs will thank you.
Writing these layers to be scalable and high-performance can be challenging, but it looks like Apple is actively doing it and willing to release the results to the rest of us. This also suggests that their previous open-sourcing of the MongoDB-compatible document layer wasn't a one-off fluke. All of this is very good news for everybody who needs to run databases in the real world.
Full disclosure: I worked on FoundationDB a long, long time ago.