News for hackers. Stuff that matters.

A comment on Benchmarking LLM Inference Back Ends: VLLM, LMDeploy, MLC-LLM, TRT-LLM, and TGI.

Re:Benchmarking LLM Inference Back Ends: VLLM, LMDeploy, MLC... by ssheng on Thursday June 06, 2024 @04:25PM (#40601960)
Hello! We're the authors of this blog post. Please let us know if there are other models and inference backends you'd like us to benchmark next.
[ Parent | on HN ]

There are no comments on this comment yet.