News for hackers. Stuff that matters.

A comment on Nvidia Dynamo: A Datacenter Scale Distributed Inference Serving Framework. Parent comment

Re:Nvidia Dynamo: A Datacenter Scale Distributed Inference S... by bytesandbits on Friday March 21, 2025 @06:04PM (#43441240)
triton is not that bad, TensorRT will give you nightmares
[ Parent | on HN ]
Re:Nvidia Dynamo: A Datacenter Scale Distributed Inference S... by dlewis1788 on Tuesday April 01, 2025 @12:50PM (#43548948)
100% - probably why vLLM is now the default back-end in Dynamo.
[ Parent | on HN ]