News for hackers. Stuff that matters.
Re:Apache Arrow Flight: A Framework for Fast Data Transport by jumpingmice on Tuesday October 15, 2019 @02:21PM (#21261841)
More people should try high performance services with non-traditional protobuf implementations. The fact that every language has a generated parser in no way preclude you from parsing them yourself. Hand-rolled serialization of your outbound messages can also be really fast, and the C++ gRPC stack will just accept preformatted messages and put them on the wire. Finally the existence of gRPC itself should not make you feel constrained against implementing the entire protocol yourself. It’s just HTTP/2 with conventional headers.
Re:Apache Arrow Flight: A Framework for Fast Data Transport by wesm on Tuesday October 15, 2019 @03:01PM (#21262354)
To be clear for anyone reading, we're parsing and generating the data-related protobufs ourselves, and retaining ownership of the memory returned by gRPC to obtain zero copy.

The C++ details are found in

https://github.com/apache/arrow/blob/master/cpp/src/arrow/fl...

Re:Apache Arrow Flight: A Framework for Fast Data Transport by jumpingmice on Tuesday October 15, 2019 @04:18PM (#21263240)
Have you considered/tried using the new ParseContext instead of CodedInputStream? It is performance-oriented.

Edit: Apparently it's also the default in protobuf 3.10

Re:Apache Arrow Flight: A Framework for Fast Data Transport by wesm on Tuesday October 15, 2019 @05:38PM (#21264150)
I wasn't aware of it but will take a look. Thanks!