News for hackers. Stuff that matters.
Re:Show HN: GPU-Accelerated Inference Hosting by sjnair96 on Tuesday July 13, 2021 @03:01AM (#27818263)
Looks awesome. Do you know if and how you guys support NVIDIA's software. For my project the NVIDIA software I'm using states it needs:

CUDA 11.3.0

cuBLAS 11.5.1.101

cuDNN 8.2.0.41

NCCL 2.9.6

TensorRT 7.2.3.4

Triton Inference Server 2.9.0

I'm new to deploying to production inference so I'm not sure if those are easily portable across such platforms or not really.

Re:Show HN: GPU-Accelerated Inference Hosting by theo31 on Wednesday July 14, 2021 @08:46PM (#27840280)
Those frameworks are installed by default in our custom environment. There is no additional setup/configuration required from you.