Re:Apple Open-Sources FoundationDB Record Layer
by ryanworl on Tuesday January 15, 2019 @08:58AM
(#18911121)
FDB itself is written in Flow, which is 98% C++ with a few extra keywords thrown on top that gets compiled to actual C++. This gives them actor model concurrency in C++ that is completely deterministic by generating very ugly code that uses callbacks instead of threads or coroutines.
The reason for Java being used here, if I had to guess, is they have experience running Java in production and the Java bindings are the most mature (in my opinion). Writing the record layer in C++ would probably be overkill. The reason the Java bindings are the most mature is probably because they needed to be to write this :)
Re:Apple Open-Sources FoundationDB Record Layer
by teacpde on Tuesday January 15, 2019 @12:27PM
(#18912907)
> Writing record layer in C++ would be overkill
Mind to elaborate on why C++ would be overkill? I understand the layered architecture decouples data model from underlying storage so it can be implemented in whatever language. To me, Flow and not having GC makes C++ a good choice.
Re:Apple Open-Sources FoundationDB Record Layer
by ryanworl on Tuesday January 15, 2019 @01:25PM
(#18913508)
It would be a fine choice, (and the document layer is) but the most performance sensitive parts of the entire system when you take it as a whole are already written in Flow. Writing it in C++ entirely would take a lot longer.
Re:Apple Open-Sources FoundationDB Record Layer
by zepto on Tuesday January 15, 2019 @11:19AM
(#18912242)
There was no gap.
Also, it’s just as possible that open sourcing it earlier could have just bogged down the decision making because of alignment issues and caused it to fail.
Open sourcing is great and the more companies like Apple do it the better, but assuming that earlier is always better doesn’t seem justified.
Re:Apple Open-Sources FoundationDB Record Layer
by neurobashing on Tuesday January 15, 2019 @09:34AM
(#18911328)
Apple's server-side WebObjects stuff is all written in Java; and IIRC, the original Control Panel for OS X was in Java using the ObjC/Cocoa bridge stuff.
Re:Apple Open-Sources FoundationDB Record Layer
by bluejekyll on Tuesday January 15, 2019 @09:55AM
(#18911481)
It still makes me sad that Apple dropped the Java Cocoa bindings. That could have been a really great thing for cross platform development (probably the reason).
Re:Apple Open-Sources FoundationDB Record Layer
by tylerchr on Tuesday January 15, 2019 @12:29PM
(#18912941)
This was a little disappointing to me; the paper[0] pitches as a design principle it’s efficiency and embeddability but it’s not clear to me how one could achieve this except in another JVM-based language. This is in contrast to the core fdb client library, which is ~easy to embed all over as it’s in C.
So it seems like services in languages like Go, Rust, python, node, etc are essentially forced to dream up a bespoke wire protocol and revert to the client-server approach. Am I missing something?
Re:Apple Open-Sources FoundationDB Record Layer
by rdtsc on Tuesday January 15, 2019 @03:32PM
(#18914801)
I think the idea is that this layer could be exposed via a networked API. Though now there would be another layer of services to manage with all the associated, deployment, security, operability issues.
Re:Apple Open-Sources FoundationDB Record Layer
by karmakaze on Thursday January 17, 2019 @01:51PM
(#18933384)
It wasn't directly stated as the reason for this choice, but being a heavy user of Cassandra and migrating away from it likely played a factor. It may have been that they had already developed a lot of supporting functions around Cassandra in Java and naturally led to the development of Record Layer in same.
Re:Apple Open-Sources FoundationDB Record Layer
by SquareWheel on Tuesday January 15, 2019 @10:27AM
(#18911771)
I'm surprised to see Apple using Github; particularly now that Microsoft owns it. It is where the developers are though. And it certainly beats a hidden download on opensource.apple.com.
Re:Apple Open-Sources FoundationDB Record Layer
by dwaite on Tuesday January 15, 2019 @11:21AM
(#18912253)
The opensource.apple.com site is really more for software that they snapshot and release (open source, not open process). They've had other projects for a long time developed in the open, and used to run https://www.macosforge.org to host them.
Some of those were acquired (either as a core developer in the case of CUPS, a codebase in the case of WebKit) but I'm not sure of other examples of open sourcing an acquired technology like this.
Re:Apple Open-Sources FoundationDB Record Layer
by jwildeboer on Tuesday January 15, 2019 @09:47AM
(#18911423)
TL;DR to save y'all looking for it: Released under Apache 2.0 License, see file ACKNOWLEDGEMENTS for licenses of included 3rd party code (some BSD style, some more ASL 2.0)
HN Announcement comments: https://news.ycombinator.com/item?id=18906341
The reason for Java being used here, if I had to guess, is they have experience running Java in production and the Java bindings are the most mature (in my opinion). Writing the record layer in C++ would probably be overkill. The reason the Java bindings are the most mature is probably because they needed to be to write this :)
Mind to elaborate on why C++ would be overkill? I understand the layered architecture decouples data model from underlying storage so it can be implemented in whatever language. To me, Flow and not having GC makes C++ a good choice.
https://techcrunch.com/2015/03/24/apple-acquires-durable-dat...
Also, it’s just as possible that open sourcing it earlier could have just bogged down the decision making because of alignment issues and caused it to fail.
Open sourcing is great and the more companies like Apple do it the better, but assuming that earlier is always better doesn’t seem justified.
I don't think it was open in the first place, was it?
1) https://en.wikipedia.org/wiki/Enterprise_Objects_Framework
So it seems like services in languages like Go, Rust, python, node, etc are essentially forced to dream up a bespoke wire protocol and revert to the client-server approach. Am I missing something?
[0] https://www.foundationdb.org/files/record-layer-paper.pdf
And a copy of the original SQL layer: https://github.com/jaytaylor/sql-layer
Some of those were acquired (either as a core developer in the case of CUPS, a codebase in the case of WebKit) but I'm not sure of other examples of open sourcing an acquired technology like this.
There's also: https://github.com/apple/foundationdb
Less risk of it being acquired by some unknown company in the future or collapsing through mismanagement.