News for hackers. Stuff that matters.
Re:Llama.cpp: Full CUDA GPU Acceleration by marcyb5st on Tuesday June 13, 2023 @04:10AM (#36307130)
I think there are several reasons.

Firstly, nVidia has been at it much longer. Just because of this tools on nVidia side feel easier to set up / are more polished (at least that was my feeling when fiddling with ROCm like a year ago).

Second but still related to #1, from the beginning even consumers nVidia cards were able to run CUDA and this made so that hobbyist and prosumers/researchers on a budget bought nVidia cards compounding even further the time/tooling advantage nVidia had. I.e. a huge user base of not only gamers but people that use their cards to do other things than gaming and know that things work on these cards.

These are, IMHO, the main reasons why everyone targets CUDA and explain why frameworks like Tensorflow or Pytorch targeted it as a first class citizen.

Re:Llama.cpp: Full CUDA GPU Acceleration by rapsey on Tuesday June 13, 2023 @04:23AM (#36307226)
If AMD was software competent the frameworks would support their drivers just as well. No one wants a monopoly.
Re:Llama.cpp: Full CUDA GPU Acceleration by marcyb5st on Tuesday June 13, 2023 @04:30AM (#36307302)
Agreed. Sorry if I gave the impression of being pro-nVidia. I am not.

But the reality is that when Tensorflow and Pytorch came to be there was no alternative. Now you need to jump through hoops to make it work with non CUDA hardware.

Additionally, while drivers play a role, I think the main difference is in the computing libraries (CUDA vs ROCm)