Re:NVIDIA introduces TensorRT-LLM for accelerating LLM infer...
by minimaxir on Friday September 08, 2023 @06:30PM
(#37440123)
Normally it's architectural changes such as FP8 support as noted in the submission, but in that case newer mid-range GPUs like the A10 and the L4 (https://www.nvidia.com/en-us/data-center/l4/) should be supported but aren't.
[ Parent
| on HN ]
There are no comments on this comment yet.