Re:Don't Use Protobuf for Telemetry
by janders on Wednesday December 30, 2020 @02:41PM
(#25585169)
I want to be a fan of Apache Avro (https://avro.apache.org/) so much for situations such as this. And while I like Avro as a standard, most of the implementations I have found (specifically in C/C++) are... lacking. I feel Avro would be a great fit for something like this close to zero overhead (assuming a pre-shared schema) but there is little to no support for pre-shared schema and the RPC part of the standard is not a great fit for telemetry. Maybe one day I'll make an Avro library I like, or contribute to an extant one.
[ Parent
| on HN ]
I think AVRO is fairly well designed overall, but JSON schemas? If you need deeply nested objects just look away, it becomes unintelligible when trying to understand the schema with any real nesting. The company I was at used it fairly well as long as you keep things 1 or 2 levels nested at most, but we had a legacy schema that was 5 or 6 levels deep at some points and it was just a disaster. My anecdote anyways.
Oh yeah. Schema nesting is a problem. I think a lot of it would be solved by having a schema allow a "types" field which just has a list your named types that can be referenced later in the list and in the actual schema. But yeah, totally concur on that pain point.