Re:Why We’re Switching to gRPC
by anderspitman on Saturday June 01, 2019 @03:30PM
(#20071055)
Interesting. I do tend to favor protocols that are a bit lower level and more flexible for composing higher-level functionality. Though this does sound pretty complicated to implement using only what capnproto offers right now. Would there be a way to jury rig "request(n)" backpressure as described in reactive streams[0] (also implemented by RSocket[1]) on top of capnproto? That's what I'm using for omnistreams and it's proven very simple to reason implement and reason about.
[0] https://github.com/reactive-streams/reactive-streams-jvm
Re:Why We’re Switching to gRPC
by anderspitman on Saturday June 01, 2019 @03:31PM
(#20071059)
Actually the more I think about it I don't think it would work, since request(n) assumes the producer can send multiple messages for each request.
Sure it does. You can implement "streaming" in Cap'n Proto by introducing a callback object, and making one RPC call for each item / chunk in the stream. In Cap'n Proto, "streaming" is just a design pattern, not something that needs to be explicitly built in, because Cap'n Proto is inherently far more expressive than gRPC.
That is, you can define a type like:
Then you can define streaming methods like: Admittedly, this technique has the problem that the application has to do its own flow control -- it has to keep multiple calls in-flight to saturate the connection, but needs to place a cap on the number of calls in order to avoid excess buffering. This is doable, but somewhat inconvenient.So I am actually in the process of extending the implementation to make this logic built-in:
https://github.com/capnproto/capnproto/pull/825
Note that PR doesn't add anything new to the RPC protocol; it just provides helpers to tell the app how many concurrent calls to make.