Re:Hidet: A Deep Learning Compiler for Efficient Model Serving
by kookamamie on Friday April 28, 2023 @02:08AM
(#35738019)
I more relevant benchmark would be against TensorRT, the current kind of model serving performance on NVIDIA GPUs.
[ Parent
| on HN ]
Re:Hidet: A Deep Learning Compiler for Efficient Model Serving
by lamchob on Friday April 28, 2023 @04:38AM
(#35738936)
In the paper (https://dl.acm.org/doi/10.1145/3575693.3575702) there is also a TensorRT comparison. There, they are pretty much neck and neck. Sometimes faster, sometimes slower.
Re:Hidet: A Deep Learning Compiler for Efficient Model Serving
by gpekhimenko on Friday April 28, 2023 @08:04AM
(#35740257)
+1. There are multiple comparisons in the paper, and we plan to provide a few more soon. The typical story that if the model is at least a bit unusual, not overly optimized manually, Hidet can provide some speedup even over TensorRT.