Re:Cuda.jl v3.3: union types, debug info, graph APIs
by xvilka on Sunday June 13, 2021 @11:00PM
(#27498817)
I wish more attention would be towards open source alternatives for CUDA, such as AMD's ROCm[1][2] and Julia framework using it - AMDGPU.jl[3]. It's sad to see so many people praise NVIDIA which is the enemy of open source, openly hostile to anything except their oversized proprietary binary blobs.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by jpsamaroo on Monday June 14, 2021 @07:59AM
(#27501552)
AMD has done great work in a very short amount of time, but let's not forget that they're still very new to the GPU compute game. The ROCm stack is overall still pretty buggy, and definitely hard to build in ways other than what AMD deems officially supported.
As AMDGPU.jl's maintainer, I do certainly appreciate more users using AMDGPU.jl if they have the ability to, but I don't want people to think that it's anywhere close in terms of maturity, overall performance, and feature-richness compared to CUDA.jl. If you already have access to an NVIDIA GPU, it's painless to setup and should work really well for basically anything you want to with it. I can't say the same about AMDGPU.jl right now (although we are definitely getting there).
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by krapht on Sunday June 13, 2021 @11:32PM
(#27498993)
Most people hack on this stuff for work, and time is money. OpenCL is just a lot less productive than CUDA for most tasks. The NVidia price premium isn't big enough to make people switch over.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by andi999 on Sunday June 13, 2021 @11:57PM
(#27499155)
Agree, been doing some cuda as part of my job since 2009, and it works very smooth, you could write non trivial programs after less than one week of learning (if you know C before). When opencl got a little hype, I had a look, I didn't manage to run a simple example, and I was asking myself: do I really want to have to use all this boilerplate code? Also at least a while back there is no good fft outside of cuda, maybe that changed though.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by kwertzzz on Monday June 14, 2021 @03:29AM
(#27500199)
This is really the main problem. As far as I know, ROCm requires a quite expensive GPU for data centers (if you want to have a current GPU) which makes it quite difficult to build a community around ROCm.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by dvfjsdhgfv on Monday June 14, 2021 @03:33AM
(#27500222)
I would love to, however for practically all tasks I need only the CUDA backend is available, not ROCm - an that is of course bad not just for open source, but also for the owners of AMD GPUs. The popularity of Nvidia's solution is so overwhelming AMD would need to do something revolutionary in order to catch up (supporting Windows and macOS wouldn't hurt either).
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by up6w6 on Sunday June 13, 2021 @09:15PM
(#27498285)
A fun fact is that the GPUCompiler, which compiles the code to run in GPU's, is the current way to generate binaries without hiding the whole ~200mb of julia runtime in the binary.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by eigenspace on Sunday June 13, 2021 @07:42PM
(#27497684)
It also happens to be one of the most easy and reliable ways I know of to install CUDA on your machine. Everything is handled through the artifact system so you don't have to mess with downloading it yourself and making sure you have the right versions and such.
(Before someone complains, you can also opt out of this and direct the library to a version you installed yourself)
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by krastanov on Sunday June 13, 2021 @09:06PM
(#27498205)
Could you elaborate a bit on this? I know that a gigantic bunch of libraries are involved that is usually terrible to install but Julia does it for you in the equivalent of a python virtual-env. However, aren't there also Linux kernel components that are necessary? How are those installed?
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by KenoFischer on Sunday June 13, 2021 @10:19PM
(#27498608)
Yes, there's a kernel component which needs to be installed, but that's usually pretty easy these days, because it's usually one of
1) You're using a container-ish environment where the host kernel has the CUDA drivers installed anyway (but your base container image probably doesn't have the userspace libraries)
2) The kernel driver comes with your OS distribution, but the userspace libraries are outdated (userspace libraries here includes things like JIT compilers, which have lots of bugs and need frequent updates) or don't have some of the optional components that have restrictive redistribution clauses
3) Your sysadmin installed everything, but then helpfully moved the CUDA libraries into some obscure system specific directory where no software can find it.
4) You need to install the kernel driver yourself, so you find it on the NVIDIA website, but don't realize there's another 5 separate installers you need for all the optional libraries.
5) Maybe you have the NVIDIA-provided libraries, but then you need to figure out how to get the third-party libraries that depend on them installed. Given the variety of ways to install CUDA, this is a pretty hard problem to solve for other ecosystems.
In Julia, as long as you have the kernel driver, everything else will get automatically set up and installed for you. As a result, people are usually up and running with GPUs in a few minutes in Julia.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by ChrisRackauckas on Sunday June 13, 2021 @10:13PM
(#27498585)
CUDA.jl installs all of the CUDA drivers and associated libraries like cudnn for you if you don't have them. Those are all vendered via the Yggdrasil system so that users don't have to deal with it.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by krastanov on Monday June 14, 2021 @12:30AM
(#27499313)
CUDA.jl does not install the actual kernel driver, right? I do not really see how it can do that and the sibling comment does confirm that the kernel driver is not managed by Julia.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by kwertzzz on Monday June 14, 2021 @02:55AM
(#27500013)
Yes, you would still need to the NVIDIA kernel driver (preferably the most current one). Desktop users typically have it already installed. But the main difficulty in my opinion is to install CUDA (with CuDNN,...). Even the TensorFlow documentation [0] is outdated in this regards as it covers only Ubuntu 18.04.
The installation process of CUDA.jl is really quite good and reliable. Per default it downloads it own version of CUDA and CuDNN, or you can use a system-wide CUDA installation by setting some environment variables [1].
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by version_five on Sunday June 13, 2021 @06:56PM
(#27497368)
I last experimented with CUDA.jl a year ago, and it was very useable then. This is a good reminder to re-evaluate the Julia deep learning ecosystem. If I were working for myself I would definitely try to do more with Julia (for machine learning). Realistically, python has such an established base that it will take some time to get orgs that are already all in on python to come over.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by dnautics on Sunday June 13, 2021 @08:23PM
(#27497962)
I think it's not dumb to target Greenfield users: just installing python gpu wheels is often difficult enough that several companies exist (indirectly) because it's so difficult to do right (e.g. selling a gpu PC with that stuff preinstalled)
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by queuebert on Sunday June 13, 2021 @11:05PM
(#27498853)
I just finished setting up a new machine to run some Kaggle stuff. Both Tensorflow and PyTorch had issues with CUDA versions and dependencies that weren't immediately fixed with a clean virtualenv, while both Knet.jl and Flux.jl installed flawlessly.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by kwertzzz on Monday June 14, 2021 @03:18AM
(#27500155)
For Pytorch, I had no issues with conda. But with Tensorflow from conda, the training process just hangs (consuming 100% of CPU but no GPU resources, despite my GPUs are recognized). I got more luck with installing Tensorflow with pip. Given the fact that Tensorflow documentation does not mention conda, I wondering how well this is supported.
Re:Cuda.jl v3.3: union types, debug info, graph APIs
by jpsamaroo on Monday June 14, 2021 @08:06AM
(#27501603)
For kernel programming, https://github.com/JuliaGPU/KernelAbstractions.jl (shortened to KA) is what the JuliaGPU team has been developing as a unified programming interface for GPUs of any flavor. It's not significantly different from the (basically identical) interfaces exposed by CUDA.jl and AMDGPU.jl, so it's easy to transition to. I think the event system in KA is also far superior to CUDA's native synchronization system, since it allows one to easily express graphs of dependencies between kernels and data transfers.
[1] https://rocmdocs.amd.com/en/latest/index.html
[2] https://github.com/RadeonOpenCompute/ROCm
[3] https://github.com/JuliaGPU/AMDGPU.jl
As AMDGPU.jl's maintainer, I do certainly appreciate more users using AMDGPU.jl if they have the ability to, but I don't want people to think that it's anywhere close in terms of maturity, overall performance, and feature-richness compared to CUDA.jl. If you already have access to an NVIDIA GPU, it's painless to setup and should work really well for basically anything you want to with it. I can't say the same about AMDGPU.jl right now (although we are definitely getting there).
Julia has all the tools required to "magically" transform code into SPIR-V kernels. Couldn't `:()` syntax be used to create kernels?
If the competition to CUDA wants to be taken serioulsy then provide the tooling and polyglot support to do so.
OpenCL was stuck in their "C only with printf debugging" mentality for too long, now it is too late.
AMD ROCm still isn't available in Windows.
If I learned anything from wasting my time with Khronos stuff, was that I should have switched to DirectX much earlier.
https://github.com/JuliaGPU/GPUCompiler.jl/ https://github.com/tshort/StaticCompiler.jl/
(Before someone complains, you can also opt out of this and direct the library to a version you installed yourself)
1) You're using a container-ish environment where the host kernel has the CUDA drivers installed anyway (but your base container image probably doesn't have the userspace libraries)
2) The kernel driver comes with your OS distribution, but the userspace libraries are outdated (userspace libraries here includes things like JIT compilers, which have lots of bugs and need frequent updates) or don't have some of the optional components that have restrictive redistribution clauses
3) Your sysadmin installed everything, but then helpfully moved the CUDA libraries into some obscure system specific directory where no software can find it.
4) You need to install the kernel driver yourself, so you find it on the NVIDIA website, but don't realize there's another 5 separate installers you need for all the optional libraries.
5) Maybe you have the NVIDIA-provided libraries, but then you need to figure out how to get the third-party libraries that depend on them installed. Given the variety of ways to install CUDA, this is a pretty hard problem to solve for other ecosystems.
In Julia, as long as you have the kernel driver, everything else will get automatically set up and installed for you. As a result, people are usually up and running with GPUs in a few minutes in Julia.
[0] https://www.tensorflow.org/install/gpu [1] https://cuda.juliagpu.org/stable/installation/overview/
Edit: the site has one project per GPU type, shame there isn't one interface that works with every GPU type instead.
https://github.com/JuliaGPU/oneAPI.jl
These are both less mature than CUDA.jl, but are in active development.
> Edit: the site has one project per GPU type, shame there isn't one interface that works with every GPU type instead.
That would be https://juliagpu.github.io/KernelAbstractions.jl
However, the low-level library for AMD GPUs is more of an alpha quality in Julia.