maleadt
maleadt on Hacker News
Posted by
maleadt on Friday June 24, 2022 @09:37AM
from the juliagpu.org dept.
Yeah, see this section of the documentation:
https://juliagpu.gitlab.io/CUDA.jl/development/profiling/. CUDA.jl also supports NVTX, wraps CUPTI, etc. The full extent of the APIs and tools is available.
Source line association when using PC sampling is currently broken due to a bug in the NVIDIA drivers though (segfaulting when parsing the PTX debug info emitted by LLVM), but I'm told that may be fixed in the next driver.
Impressive! The PTX to SPIR-V compiler must have been quite a bit of work; what's the coverage of the ISA like?
With oneAPI I had hoped to get the inverse, a oneAPI implementation for NVIDIA hardware, but I don't think the CUDA driver API is low-level enough to do so (e.g. explicit vs global contexts). And yes, I know of Codeplay's implementation of DPC++ for NVIDIA GPUs, but that doesn't implement oneAPI Level0 APIs so is not usable for other languages.
You can emit `trap` or `exit` in the PTX code (although that has exposed many bugs in the PTX assembler because it does not expect that kind of often divergent control flow). But even if you'd just have the kernel return and otherwise produce invalid results, the fact that you can then report a bounds error instead of silently corrupting data and/or generating a fatal ERROR_ILLEGAL_ACCESS (requiring an application restart) is a significant usability improvement.
Yes, that's fair. I focused on CUDA.jl because it is the most mature, easiest to install, etc. but as I mentioned we're actively working on generalizing that support as much as possible, and as a result support for AMD (AMDGPU.jl) and Intel (oneAPI.jl) GPUs is rapidly catching up.
Since we use fat array objects, and not raw pointers, we know the size of the array and can perform bounds checks at run time. We then have a mechanism to throw an exception and signal it to the CPU to display it there. That's obviously quite expensive, so you can disable it with that annotation (the Julia debug setting also controls the granularity, and thus how expensive the exception handling is). It's fairly primitive, i.e. no full-featured exception handling (for now), but has proven very useful already.
The Julia array abstractions make it so that most code is vendor-neutral already, and you execute on whatever GPU back-end you want by using an appropriate array type. For vendor-neutral kernel programming there's GPUArrays.jl and KernelAbstractions.jl, but both aren't currently very user friendly (but are actively used as a building block for user-facing applications and APIs).
I'm looking at targeting it from Julia, and the lower-level (Level Zero) API seems rather nice, resembling the CUDA driver API but building on SPIR-V. It's also nice how the API is decoupled from the implementation, so let's hope more vendors implement it (apparently Codeplay is working on a CUDA-based implementation).
Great talk! Any thoughts on Intel's oneAPI?
Our view is that to get performance out of a system (here CUDA), it's better not to start abstracting it right away. So we have CUDAnative.jl and CUDAdrv.jl for fairly low-level CUDA programming, albeit in a high-level language. However, with CuArrays.jl we implement the Julia array interface for CUDA GPUs. That means you can write array code for one platform (CPU using Base.Array) and start using hardware accelerators by just switching the array type (CUDA GPU using CuArray). Of course, real-life applications might still need to use CUDA specific functionality for one reason or another, but at least you can get most of the way without platform-specific programming.
Author here, happy to answer any questions! We've been developing and maintaining this toolchain for a while now, so the relevant packages (CUDAnative.jl for kernel programming, CuArrays.jl for a GPU array abstraction) are much more mature. Our focus has recently been on implementing a common base of array operations that can be used across devices (GPU, CPU, etc), so that users can develop using the base CPU array type, quickly benefit from a GPU by switching to CuArrays, only to rely on specific CUDA-specific functionality from CuArrays/CUDAnative when they need custom functionality.
As the author of the underlying framework: the dependency on CUDA is unfortunate indeed, but it was the only viable option at the time. OpenCL tooling was (is) way more fragmented, there's no unified compilation target or back-end, differences between vendors, etc. However, much of this work has dealt with figuring out how to compile Julia for accelerators while integrating with the existing compiler, all of which is transferable to future back-ends. Once the SPIR-V back-end is better integrated with upstream LLVM, we will try to target that toolkit instead.
But it relies on Pocket successfully extracting the content. Plenty of sites yielded a broken ereader version on my Kobo (mainly missing or unreadable figures).
A colleague of mine mentioned how this has been the case for quite a bit longer than Haswell, see "Compiler mitigations for time attacks on modern x86 processors" (from 2012). Or Agner Fog's timings pages, where many DIV instructions list widely ranging latencies, with the comment "[l]ow values are for small results, high values for high results".
It is definitely possible to do that in Julia. The reason a didn't yet is purely a manner of priorities, I first focused on wrapping the basic primitives (calling a kernel, marshalling arguments, etc) in a user-friendly way.
OpenCL.jl is purely the runtime part, ie. it still requires you to write manual OpenCL code, after which you can use the julia wrapper to manage that code.
My project also provides compiler support for lowering Julia to CUDA assembly, so you don't need to write CUDA code yourself. Added to that, my runtime also contains (PoC) higher-level wrappers, making it easier to call CUDA kernels, upload data, etc.
Concerning tying yourself to the NVIDIA-stack: it's still the most mature and versatile toolchain, which is why I picked it in the first place. My long term plan was to switch over to SPIR (or some other cross-vendor stack) as soon as possible. At that point, switching user-code over to that new back-end would (theoretically) not require that much effort, since the kernels are written in julia-code instead of CUDA C (except for the runtime interactions, of course).
Thanks for the kind words! This was exactly what I was aiming for: the code (or insights) to be reusable without too much hassle. More so because part of it was developed in the scope of my PhD; I wouldn't want to know how many failed or unpublishable research results are stowed away on some grad student's computer.
Wouldn't it be interesting to make the page editable, for example in typical wiki-style like on kernelnewbies.org? Since much of your reader base consists of LLVM developers I'm sure this could further improve the content, as well as make it easy to fix small things such as typos.
Carmack has said somewhere (I can't retrieve the source right now) that an alternative, slower method will be used in the GPL'd codebase, effectively working around Creative's patent claims.
But you're right about the subreddits, and this is one of the more important features which manages to keep acquainted users from leaving the site, even if they are tired of the now 4chan-esque frontpage.