Re:Deploying Transformers at Scale
by pthaine on Tuesday February 22, 2022 @06:50PM
(#30435541)
Key takeways: (1) ONNXRuntime is the best inference package for Transformer networks; (2) Nvidia Triton, together with ONNXRuntime is the best solution for GPU inference; (3) Optimization matters. Itβs quite easy to unlock a >10X performance gain in 2022.
[ Parent
| on HN ]