News for hackers. Stuff that matters.

A comment on Deploy Gemma 7B with TensorRT-LLM and achieve > 500 tok/s. Parent comment

Re:Deploy Gemma 7B with TensorRT-LLM and achieve > 500 tok/s by v1vek on Friday February 23, 2024 @05:27PM (#39486953)
the latency to serve the end-user esp with chained calls impacts ux quality
[ Parent | on HN ]

There are no comments on this comment yet.