News for hackers. Stuff that matters.
Re:The Sandstorm Team Is Joining Cloudflare by cstrahan on Monday March 13, 2017 @04:04PM (#13861828)
I'm pretty stoked about future cap'n proto developments. In the scenarios where I've brought up Cap'n Proto, all but the most senior/prolific developers have a near impossible time of ascertaining why one might want to use capnp over competing approaches (e.g. Protobuf). I think, despite a pretty decent description on the website and my own skills in explaining/teaching, such protocols are nearly impossible for the majority of developers to intuit -- so they're all perceived as being nearly identical, save for financial backing and other forms of social proof.

With some of the rough corners filed smooth (e.g. Windows support), a bit more marketing and promotion, and eventually more big names advertising their use of capnp, my job getting buy in from my peers would be much easier.

Re:The Sandstorm Team Is Joining Cloudflare by hueving on Monday March 13, 2017 @05:19PM (#13862523)
Can you provide a 'top 3' list of reasons to use Cap'n Proto over Protobufs?
Re:The Sandstorm Team Is Joining Cloudflare by doh on Monday March 13, 2017 @05:32PM (#13862628)
1) Cap'n Proto doesn't encode/decode messages thus it's nuch cheaper for processing and memory management

2) protobuf in the proto3 design doesn't cary default values. So if you have a bool field and want to explicitly send false, well you have to change it to some other type or use the default values all the time

3) protobuf generates incredibly large serialization/deserialization support coce for each template. For some languages like Python in can be in hundreds of kilobytes. Cap'n proto messages are significantly smaller

There is more for CnP but Protobuf has much better support and is by default used in projects like gRPC. Also new CnP is lacking speed in new development in comparison to Protobuf.

But I'm using in one of my side projects and I'm very happy with it

Re:The Sandstorm Team Is Joining Cloudflare by euyyn on Monday March 13, 2017 @10:17PM (#13864423)
(2) is explicitly one of the reasons to use proto3 over proto2, though :) It's weird seeing it listed as a disadvantage instead of the other way around.
Re:The Sandstorm Team Is Joining Cloudflare by doh on Tuesday March 14, 2017 @02:07AM (#13865401)
Depends on the use case. For instance if you're mapping the data to database, you can't send null variables. Or you store the defaults (waste of space) or you send it as another type.

In our case we've now over 1T of rows. Storing default variables would be incredibly inefficient and expensive.

Re:The Sandstorm Team Is Joining Cloudflare by euyyn on Tuesday March 14, 2017 @06:40PM (#13872149)
I'm not understanding your case much. Proto3 does not serialize zeros/false, precisely because it'd be a waste without a benefit. After deserialization, the value is as equal to zero as if the zero had been written on the wire or on disk. Why would you want to store that?

Is nullability of primitive types what you're missing? As in, having a boolean value that can be true, false, or null. If you need that, defining your field as google.protobuf.BoolValue instead of bool gives you that (same with the other primitive types: https://github.com/google/protobuf/blob/master/src/google/pr... ). But making primitive types never null aligns proto3 with most programming languages, making the generated code more idiomatic and performant.

Re:The Sandstorm Team Is Joining Cloudflare by hueving on Monday March 13, 2017 @05:44PM (#13862727)
Thanks. Those are pretty significant advantages. These things should be on the home page of the project. That's how smaller projects gain share.
Re:The Sandstorm Team Is Joining Cloudflare by andrewstuart2 on Monday March 13, 2017 @06:47PM (#13863250)
The first one is, but as the obviously hyperbolic "infinitely faster" claim.
Re:The Sandstorm Team Is Joining Cloudflare by nurettin on Tuesday March 14, 2017 @01:00AM (#13865134)
I use protobuf-generated RTTI for the purpose of doing endpoint SQL storage of the messages. (tables, columns, foreign keys, insert/select statements are all auto-generated from the protobuf message) using C++.

Does capnproto provide similar RTTI information to walk through properties or recurse into other messages?

Re:The Sandstorm Team Is Joining Cloudflare by kentonv on Tuesday March 14, 2017 @02:00AM (#13865364)
Absolutely. I was the one who originally designed the Protobuf RTTI (aka "reflection") interface, after all... :)

It's actually considerably easier to implement for Cap'n Proto since the in-memory objects are actually backed by byte buffers containing the wire format. No need to compute offsets of class members (which technically violates the C++ standard though it works on all compilers).

Re:The Sandstorm Team Is Joining Cloudflare by skybrian on Monday March 13, 2017 @05:49PM (#13862767)
re: "Doesn't encode/decode messages"

Does this mean that individual fields are encoded/decoded lazily?

Re:The Sandstorm Team Is Joining Cloudflare by kentonv on Monday March 13, 2017 @06:08PM (#13862918)
Well, you could look at it that way. But the "encoding" is the same encoding that an "int" field in a struct uses, and "decoding" it is a trivial load instruction.
Re:The Sandstorm Team Is Joining Cloudflare by lastofus on Monday March 13, 2017 @06:57PM (#13863339)
At what point is byte ordering handled?
Re:The Sandstorm Team Is Joining Cloudflare by kentonv on Monday March 13, 2017 @07:12PM (#13863444)
The format is little-endian, so it's necessary to use a little-endian load instruction, which almost all architectures (including most big-endian architectures) have.

(But every widely-used arch these days is LE.)

Re:The Sandstorm Team Is Joining Cloudflare by StreamBright on Monday March 13, 2017 @05:08PM (#13862394)
Performance might be one reason to use Cap'n Proto over Protobuf.

http://dbeck.github.io/5-lessons-learnt-from-choosing-zeromq...