News for hackers. Stuff that matters.

A comment on NVIDIA introduces TensorRT-LLM for accelerating LLM inference on H100/A100 GPUs. Parent comment

Re:NVIDIA introduces TensorRT-LLM for accelerating LLM infer... by boxfire on Friday September 08, 2023 @06:59PM (#37440337)
3090 does 4x nvlink each way, ~100 GB/s bidirectional data
[ Parent | on HN ]
Re:NVIDIA introduces TensorRT-LLM for accelerating LLM infer... by why_only_15 on Friday September 08, 2023 @07:03PM (#37440364)
Oh huh I didn't know about that!
[ Parent | on HN ]