News for hackers. Stuff that matters.
Re:Zero-Shot Text Classification on a low-end CPU-only machine? by backend-dev-33 on Wednesday October 09, 2024 @04:54AM (#41785831)
> additional challenge of offering it over a network socket across multiple requests and doing so in a performant manner.

@kkielhofner thanks a lot! now I realize it. I see, there is even GRPC support in Triton, so it make sense.

Re:Zero-Shot Text Classification on a low-end CPU-only machine? by kkielhofner on Friday October 11, 2024 @10:36PM (#41815907)
Make sure to check out the existing Triton client libraries:

https://github.com/triton-inference-server/client