News for hackers. Stuff that matters.

A comment on Is LMDeploy the Ultimate Solution? Why It Outshines VLLM, TRT-LLM, TGI, and MLC.

Re:Is LMDeploy the Ultimate Solution? Why It Outshines VLLM,... by timliu9 on Thursday June 20, 2024 @11:50AM (#40740092)
Why was onnx not part of the tested runtimes? Seems like an oversight
[ Parent | on HN ]
Re:Is LMDeploy the Ultimate Solution? Why It Outshines VLLM,... by chaoyu on Thursday June 20, 2024 @12:06PM (#40740279)
onnx is not a good option for LLM type of autoregressive generation
[ Parent | on HN ]
Re:Is LMDeploy the Ultimate Solution? Why It Outshines VLLM,... by helloericsf on Thursday June 20, 2024 @12:23PM (#40740442)
Personally, I never seen onnx used for LLM.
[ Parent | on HN ]