Llama.cpp: Full CUDA GPU Acceleration

Re:Llama.cpp: Full CUDA GPU Acceleration by sorenjan on Tuesday June 13, 2023 @06:31AM (#36308117)

I'm hopeful for SYCL [0] to become the cross platform alternative, but there doesn't seem to be a lot of uptake from projects like this, so maybe my hope is misplaced. It's an official Khronos standard, and Intel seems to like it [1], but neither of those things are enough to change things.

Someone who knows about this space that can comment on the likelihood that SYCL will be a good option eventually? Cross platform and cross vendor compatibility would be really nice, and not supporting the proprietary de facto standard would also be a bonus as long as the alternative works well enough.

[0] https://www.khronos.org/sycl/

[1] https://spec.oneapi.io/versions/latest/elements/sycl/source/...

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by mschuetz on Tuesday June 13, 2023 @07:51AM (#36308869)

> It's an official Khronos standard

I think that's the problem. Khronos isn't known for good UX, and being from Khronos is exactly the reason why I'm not even bothering to check it out. I want an alternative to CUDA, but I also want it to be as easy to use as CUDA.

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by LoganDark on Tuesday June 13, 2023 @07:56AM (#36308926)

Being from Khronos is also a reason why it might actually be usable in a decade's time, like Vulkan.

(Vulkan is from 2015 and is just recently starting to become usable.)

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by sicariusnoctis on Saturday June 17, 2023 @02:05AM (#36367724)

From a non-expert's standpoint, Vulkan feels quite unusable and complex.

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by mschuetz on Tuesday June 20, 2023 @05:51AM (#36401764)

From an expert's standpoint, it's still quite unusable and complex.

[ Parent | on HN ]