News for hackers. Stuff that matters.

A comment on NVIDIA introduces TensorRT-LLM for accelerating LLM inference on H100/A100 GPUs. Parent comment

Re:NVIDIA introduces TensorRT-LLM for accelerating LLM infer... by spott on Friday September 08, 2023 @07:09PM (#37440422)
The weights of gpt3 weren’t released, but the architecture is pretty close to gpt2 and explained in the paper: https://arxiv.org/pdf/2005.14165.pdf

I think the “ready to run” might be more of a byow (“bring your own weights”) kinda thing.

[ Parent | on HN ]

There are no comments on this comment yet.