News for hackers. Stuff that matters.

A comment on Nvidia Dynamo: A Datacenter Scale Distributed Inference Serving Framework. Parent comment

Re:Nvidia Dynamo: A Datacenter Scale Distributed Inference S... by ipsum2 on Tuesday March 18, 2025 @06:54PM (#43406165)
Ray doesn't offer anything if you use vLLM on top of Ray Serve though.
[ Parent | on HN ]
Re:Nvidia Dynamo: A Datacenter Scale Distributed Inference S... by dzr0001 on Tuesday March 18, 2025 @08:37PM (#43407037)
It does if you need pipeline parallelism across multiple nodes.
[ Parent | on HN ]