Re:Bebop v3: a fast, modern replacement for Protocol Buffers
by andrew_rfc on Tuesday March 12, 2024 @12:53PM
(#39681803)
Apologies if my initial comment came off the wrong way; when I say “not alternatives” I was injecting bias into that statement since I normally use Cap’n Proto on embedded and memory constrained systems (amazing library btw.)
When I created Bebop years ago I tried to benchmark it against Cap’n Proto, but the lack of a semi-decent web or C# implementation mentally made me recategorize.
So I suppose I’m just comparing what’s closest in terms of ecosystem support rather than a purely functional level.
Re:Bebop v3: a fast, modern replacement for Protocol Buffers
by kentonv on Tuesday March 12, 2024 @01:06PM
(#39682003)
Yeah that's fair, it does seem like Bebop and capnp are targeted at different use cases, even if they're both Protobuf alternatives.
It obviously doesn't make sense to benchmark a JavaScript Bebop implementation against a C++ Cap'n Proto implementation, nor does it make sense to compare against a JS Cap'n Proto implementation written by a third-party contributor that is incomplete and unmaintained... so there isn't really a sensible comparison to make.
The main limitation of Cap'n Proto compared to Protobuf is the ecosystem -- missing or poor-quality implementations in many languages, limited tooling, etc. Admittedly this is probably a showstopper for most users. It's also the hardest thing for any new contender to solve.
With all that said I would tend to agree that benchmarks are probably pointless. I've spent a lot of time benchmarking serialization and one thing I know is that benchmark results will vary wildly depending on the use case. A benchmark of an example/toy use case isn't really indicative of performance in a real use case.
(I'm the author of Cap'n Proto. I don't know much about Flatbuffers so can't comment there.)