Re:gRPC for Microservices Communication
by brinox on Thursday September 23, 2021 @05:14PM
(#28635526)
You can configure the message size limit on your gRPC server, doesn't that fix your issue?
Re:gRPC for Microservices Communication
by CommonGuy on Thursday September 23, 2021 @05:21PM
(#28635603)
Another option could be to "chunk" the messages with client-/server-streaming or bi-directional calls. But if you call your API from a browser, that may not be possible yet
Re:gRPC for Microservices Communication
by sjnair96 on Friday September 24, 2021 @01:56AM
(#28639078)
Yes, but that would involve us digging into the code of the gRPC server, which is unfortunately closed to us :(
Re:gRPC for Microservices Communication
by sjnair96 on Friday September 24, 2021 @01:55AM
(#28639072)
Ahh, we don't have access to the server. It's closed - an NVIDIA inference engine. Which under the hood talks to their Triton engine. Unfortunately, while Triton allows configuring the limit, the layer in front of it eats our channel options which have the message size configurations.
Recently, I was working on building the backend for our Speech Recognition engine and it seems like the message size limit came in the way of being able to perform batch/offline inference. I might be lacking some of the intuition behind designing and deploying such services, and am curious if you guys have run into this issue. The platform doesn't seem to accept the channel options from the client.
That and the issue of no easy way to do gRPC calls from a browser yet, and the resulting architecture then needing to have been built the ground up with considerations for an extra piece - a gateway, or something like Envoy to make it more accessible over the web. I feel like that would be okay if I was working on a project with k8 ingress set up already and already utilizing microservices. But for getting running it seems like added complexity. Not hating on it or anything, just sharing my experiences and thoughts, hoping that some of you might've had experience reasoning and dealing with such designs and can offer some insight.