News for hackers. Stuff that matters.
Re:Revisiting Apple Notes (6): The Protobuf by cryptonector on Monday September 21, 2020 @12:15PM (#24545129)
Protobufs is just yet another structured data serialization format. There's nothing special to it other than "it's not one of the others". It's a bad reinvention of the wheel though. That's why there's new and better things like flatbuffers. The most important thing to take away from protobufs is that in a field full of literature and prior art, one should not ignore it.
Re:Revisiting Apple Notes (6): The Protobuf by mumblemumble on Monday September 21, 2020 @12:35PM (#24545400)
For the curious, here's my hot take on the details:

Protocol buffers, being older, are far and away the more widely supported format. If you're looking for something like a wire format for RPC, or usability from basically any common language is important to you, protobuf will be the path of least resistance. However, there are some design warts to work around. The most notable one (IMO) is that varints make dealing with numbers a rather delicate subject.

Flatbuffers are similar, but made some design decisions allow for a zero-copy implementation. This can be good for performance. Overall, flatbuffers have always struck me as being promising, but language and tooling support isn't nearly as comprehensive, which has always kept me from giving them a serious try at work.

Re:Revisiting Apple Notes (6): The Protobuf by mumblemumble on Monday September 21, 2020 @05:15PM (#24548568)
A much later thought, that perhaps speaks to the idea that one size does not fit all:

The Apache Arrow project, which is the most ambitious project I know of for sharing data among systems in a performant way, ended up going with a mix of options. Actual data is stored in a project-specific format. When the data are serialized (e.g., to disk or for transport over the wire), headers, schemata, and suchlike are formatted using FlatBuffers. The RPC protocol itself uses Protobuf for the messages.

Re:Revisiting Apple Notes (6): The Protobuf by kortex on Monday September 21, 2020 @11:02PM (#24550452)
Protobufs aren't perfect, but flatbuffers aren't the holy grail, either. I've found the Protobuf ecosystem way better documented (and yet still lacking), and way more market penetration (due to grpc, no doubt). I've found flatbuf's api ergonomics absolutely dreadful. The python flatbuf library re-parses bytes on each access, which kind of defeats the purpose of zero copy. Maybe it's better in other langs, but python was where I was hoping to reap performance gains and didn't see them.