Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

Re:Intel Gaudi2 chips outperform Nvidia H100 on diffusion tr... by yukIttEft on Monday March 11, 2024 @01:14PM (#39670837)

I'm wondering how AI scientists work these days. Do they really hack Cudakernels or do they plug models together with highlevel toolkits like pytorch?

Considering its the latter, considering pytorch takes care of providing optimized backends for various hardwares, how big of a moat is Cuda then really?

[ Parent | on HN ]

Re:Intel Gaudi2 chips outperform Nvidia H100 on diffusion tr... by david-gpu on Monday March 11, 2024 @01:23PM (#39670957)

Pytorch relies heavily on the extensive libraries of high-performance kernels provided by NVidia, such as cuDNN.

In other words, it goes something like this:

    Application
    Pytorch (and similar)
    cuDNN (and similar)
    CUDA (and similar)
    NVidia GPU

My opinion, based on what I saw those wizards do, is that reproducing the feature set and efficiency of cuDNN/cuBLAS is deeply nontrivial.

[ Parent | on HN ]