News for hackers. Stuff that matters.
Re:Don't Use Protobuf for Telemetry by haberman on Wednesday December 30, 2020 @04:37PM (#25586447)
> Google's implementations, at least C++ and Java, are a bunch of bloated crap (or maybe they're very good, but for a use case that I haven't yet encountered).

As someone who has been working on protobuf-related things for >10 years, including creating a size-focused implementation (https://github.com/protocolbuffers/upb), and has been working on the protobuf team for >5 years, I have a few thoughts on this (thoughts are my own, and I don't speak for anybody else).

I think it is true that protobuf C++ could be a lot more lean than it currently is (I can't speak to Java as I don't work on it directly). That's why I created upb to begin with. But there's also a bit more to this story.

The protobuf core runtime is split into two parts, "lite" and "full". If you don't need reflection for your protos, it's better to use "lite" by using "option optimize_for = LITE_RUNTIME" in your .proto file (https://developers.google.com/protocol-buffers/docs/proto#op...). That will cut out a huge amount of code size from your binary. On the downside, you won't get functionality that requires reflection, such as text format, JSON, and DebugString().

Even the lite runtime can get "lighter" if you compile your binary to statically link the runtime and strip unused symbols with -ffunction-sections/-fdata-sections/--gc-sections flags. Some parts of the lite runtime are only needed in unusual situations, like ExtensionSet which is only used if your protos use proto2 extensions (https://developers.google.com/protocol-buffers/docs/proto#ex...). If you avoid these cases, the lite runtime is quite light.

However, there is also the issue of the generated code size. Unlike the runtime, this is not a fixed cost, but is proportional to the number of messages you use. If you have a lot of messages it can quickly dwarf the size of the runtime. For this reason, C++ also supports "option optimize_for = CODE_SIZE" which uses reflection-based algorithms for all parsing/serialization/etc instead of using generated code. This means you pay the fixed size hit from linking in the full runtime, but the generated code size is much smaller. On the downside, "optimize_for = CODE_SIZE" has a severe ~10x speed penalty for parsing and serialization.

I have long had the goal of making https://github.com/protocolbuffers/upb competitive with protobuf C++ in speed while achieving much smaller code size. With the benefit of 10 years of hindsight and many wrong turns, upb is beginning to meet and even surpass these goals. It is an order of magnitude smaller than protobuf C++, both in the core runtime and the generated code, and after some recent experiments it is beginning to significantly surpass it in speed also (I want to publish these results soon, but the code was merged in this PR: https://github.com/protocolbuffers/upb/pull/310).

upb has downsides that prevent it from being fully "user ready" yet: the API is still not 100% stable, there is no C++ API for the generated code yet (and C APIs for protobuf are relatively verbose and painful), it has a bunch of legacy APIs sitting around that I am just on the verge of being able to finally delete, and it doesn't support proto2 extensions yet. On the upside, upb is 100% conformant on every other protobuf feature, it supports reflection, JSON, and text format, but also lets you omit these if you don't want to pay the code size.

I hope 2021 is a year when I'll be able to publish more about these results, and when upb will be a more viable choice for users who want a smaller protobuf implementation.

Re:Don't Use Protobuf for Telemetry by tijsvd on Wednesday December 30, 2020 @04:57PM (#25586639)
What is this thing with JSON support? Don't people use pb so they do not have to deal with JSON? I'd expect that for a truly lean pb implementation, adding JSON is a 300% increase in code size?
Re:Don't Use Protobuf for Telemetry by kentonv on Wednesday December 30, 2020 @05:19PM (#25586867)
People implementing public-facing APIs might not be comfortable forcing all the consumers of that API to use Protobuf, but may want to use Protobuf internally within their service. So, Protobuf lets you convert to/from JSON, so that your service can expose a JSON API while maintaining Protobuf schemas and never dealing with a JSON parser directly.

The implementation is based on Protobuf's existing reflection features, so if you are already compiling with that enabled, the JSON implementation does not add much code bloat on top of that.

Re:Don't Use Protobuf for Telemetry by joshuamorton on Wednesday December 30, 2020 @05:18PM (#25586862)
[Also a googler]:

PB is used for a wide variety of things, its a lingua-franca of data interchange within Google, but web (frontend) still often uses json. So if I want to have some proto in a database and render it on a UI to a user, somewhere in there I'm probably going to be translating the raw proto to either json or json-like (I'll admit I'm not sure how stuff like grpc-web fits in here, if you can just get proto-serialized bytes passed around in some a blob, but there are some pretty low-overhead json proto encodings that exist anyway).

Re:Don't Use Protobuf for Telemetry by tijsvd on Wednesday December 30, 2020 @06:04PM (#25587357)
Sure that makes some sense.

But it's somewhat ugly as well, from an architecture point of view. Because that argument translates to anything else you might want to do with these objects.

In past libraries for C++ I've tried to prevent this kind of coupling by adding generated template methods like "walk(f)" where f would be a templated callable, called with a descriptor and data reference for each field. Any kind of pretty printer or SQL statement can be built that way.

Re:Don't Use Protobuf for Telemetry by haberman on Wednesday December 30, 2020 @05:27PM (#25586955)
> Don't people use pb so they do not have to deal with JSON?

I think that's one reason, but there are others too, like getting generated type-safe accessors and an explicit schema. I've sometimes used protobuf for this reason even when I'm not planning to use binary format.

> I'd expect that for a truly lean pb implementation, adding JSON is a 300% increase in code size?

Here is a breakdown of a binary that uses upb and links in binary format, JSON format, text format, and some generated protos:

         VM SIZE
     ---------------
      20.6%  26.5Ki    Generated Reflection
        45.7%  12.1Ki    bazel-out/k8-opt/bin/external/com_google_protobuf/src/google/protobuf/test_messages_proto3.upbdefs.c
        39.3%  10.4Ki    bazel-out/k8-opt/bin/external/com_google_protobuf/src/google/protobuf/test_messages_proto2.upbdefs.c
         5.0%  1.33Ki    bazel-out/k8-opt/bin/external/com_google_protobuf/conformance/conformance.upbdefs.c
         3.1%     832    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/struct.upbdefs.c
         2.4%     648    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/wrappers.upbdefs.c
         1.2%     320    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/timestamp.upbdefs.c
         1.2%     312    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/duration.upbdefs.c
         1.1%     304    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/field_mask.upbdefs.c
         1.1%     296    [1 Others]
      19.1%  24.5Ki    JSON
        60.4%  14.8Ki    upb/json_decode.c
        39.6%  9.67Ki    upb/json_encode.c
      14.7%  18.9Ki    Reflection Runtime
        81.9%  15.4Ki    upb/def.c
        18.1%  3.41Ki    upb/reflection.c
      13.9%  17.8Ki    Generated Code
        46.9%  8.34Ki    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/wrappers.upb.c
        17.6%  3.12Ki    bazel-out/k8-opt/bin/external/com_google_protobuf/src/google/protobuf/test_messages_proto3.upb.c
        16.4%  2.92Ki    bazel-out/k8-opt/bin/external/com_google_protobuf/src/google/protobuf/test_messages_proto2.upb.c
        14.7%  2.61Ki    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/descriptor.upb.c
         2.0%     356    bazel-out/k8-opt/bin/external/com_google_protobuf/conformance/conformance.upb.c
         1.4%     264    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/struct.upb.c
         0.5%      88    [2 Others]
         0.3%      48    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/any.upb.c
         0.3%      48    bazel-out/k8-opt/bin/external/com_google_protobuf/google/protobuf/duration.upb.c
      10.0%  12.9Ki    Binary Format
        54.0%  6.95Ki    upb/decode.c
        46.0%  5.92Ki    upb/encode.c
       8.8%  11.3Ki    Core Runtime
        66.3%  7.52Ki    upb/table.c
        17.8%  2.02Ki    upb/upb.c
        15.9%  1.80Ki    upb/msg.c
       5.6%  7.19Ki    [34 Others]
       3.8%  4.94Ki    Text Format
       100.0%  4.94Ki    upb/text_encode.c
       3.5%  4.46Ki    [section .rodata]
     100.0%   128Ki    TOTAL
So JSON is pulling in 25Ki directly, plus it depends on the 19Ki reflection runtime and the generated reflection.

But it is all a la carte: if you don't want JSON, just don't use it and don't link it in.

(btw that 7Ki for upb/table.c is to support protobuf maps, I want to make that a la carte also so that you don't pay for it when you don't use maps).

Re:Don't Use Protobuf for Telemetry by 1f60c on Wednesday December 30, 2020 @10:51PM (#25589403)
How'd you generate that listing? It seems useful.
Re:Don't Use Protobuf for Telemetry by haberman on Thursday December 31, 2020 @12:20AM (#25589853)
I used my tool Bloaty, which parses the ELF and DWARF information in the binary: https://github.com/google/bloaty
Re:Don't Use Protobuf for Telemetry by 1f60c on Thursday December 31, 2020 @10:03PM (#25600420)
Thanks!