News for hackers. Stuff that matters.
Re:Simple Binary Encoding by haberman on Monday May 05, 2014 @09:12PM (#7702351)
This sounds very similar in concept to Cap'n Proto, and given how much hard-earned experience Kenton has and how much time he's put into it, I'd put my bets on Cap'n Proto if you're looking for something in this design space.

I haven't looked at SBE in detail, but given the high-level description I expect it would have a similar set of trade-offs vs. Protocol Buffers as Cap'n Proto does: it will have unparalleled serialize/deserialize performance (since they are basically no-ops), but sparse messages (where only a few of many defined fields are set) will be arbitrarily larger than protobufs, so a separate compression step will be necessary to be size-competitive, at which point some of the speed advantage is lost.

Re:Simple Binary Encoding by kasey_junk on Monday May 05, 2014 @10:28PM (#7702562)
Kenton Varda deserves the respect you are giving him when it comes to serialization libraries. Cap'n Proto is a great project that I'd love to work with anytime. That said, Martin Thompson whose blog this is a link to is also very well respected in his particular domain (low latency messaging systems especially on the JVM and JVM/C++ hybrid space).

All that's to say, the Cap'n Proto and SBE libraries have exceedingly similar design philosophies and pedigrees. I expect them to have near performance parity in the shared language of C++. But if you are looking for a JVM/C# solution I think you can pretty reasonably choose to start working with SBE today as it provides those language bindings out of the gate and there is unlikely to be such a performance mismatch between SBE and Cap'n Proto in the future for it to matter long term.

Re:Simple Binary Encoding by haberman on Tuesday May 06, 2014 @11:07AM (#7704909)
I can totally believe that Martin Thompson is highly capable given what I read in the article. I guess my bias is against a proliferation of similar-but-incompatible messaging formats. But I suppose that if Cap'n Proto doesn't support JVM/C#, then that doesn't leave someone on those platforms a lot of options.
Re:Simple Binary Encoding by kentonv on Tuesday May 06, 2014 @06:14PM (#7707441)
There's been a lot of false starts on getting Java support implemented (people saying they'd do it, and then not), and one or two for C# as well. I wish I had time to just sit down and crank out a Java implementation myself -- I think it would take a week or two -- but right now I really need to be putting everything into Sandstorm.io. :/

It will happen eventually, though.

Re:Simple Binary Encoding by kasey_junk on Tuesday May 06, 2014 @09:57PM (#7708218)
I think one problem with this is that a JNI solution would be "trivial" but is wholly unsatisfying (JNI being somewhat incompatible with the performance goals). The Unsafe and/or ByteBuffer solution on the other hand would be a lot of work to create and maintain (so much so that I wouldn't sign up for the job).

SBE on the other hand is targeting a space dominated by C++/Java/C#. They can basically ignore every other language binding without losing any of their target audience. It also doesn't hurt that they have a patron with a financial motivation to back them...