News for hackers. Stuff that matters.

A comment on NVIDIA introduces TensorRT-LLM for accelerating LLM inference on H100/A100 GPUs. Parent comment

Re:NVIDIA introduces TensorRT-LLM for accelerating LLM infer... by nkingsy on Friday September 08, 2023 @06:22PM (#37440065)
https://github.com/NVIDIA/TensorRT/issues/982

Looks like tensorRT does work with 30x0, but I couldn't find much.

[ Parent | on HN ]

There are no comments on this comment yet.