News for hackers. Stuff that matters.
Re:An Even Easier Introduction to CUDA by jupiter90000 on Saturday February 04, 2017 @11:09AM (#13567421)
Does anyone familiar with the state of GPU programming think OpenCL will eventually 'win' over CUDA? Although CUDA has more adoption, I don't like the idea of using it and being locked into a specific vendor. Of course nVidia is only supporting outdated versions of OpenCL for now. Am I a fool for hoping OpenCL eventually becomes the standard?
Re:An Even Easier Introduction to CUDA by se6 on Saturday February 04, 2017 @03:32PM (#13568877)
In 2013 we started GPU programming at the company I work for. We carefully evaluated CUDA and OpenCL and decided to go for OpenCL because it was a standard and we could chose between 2 vendors of GPU. I can tell you that in 2017 we do not regret our choice. It is great to be able to run our code on both AMD and NVidia GPUs, and to offer our customers to choose whichever GPU vendor they prefer.

Many people criticise OpenCL because when you come from C++ it seems a lot of work. It is true that OpenCL has an API influenced by OpenGL and is verbose. However it is not difficult to write a small framework specific to your needs and domain to factorise much of this verbosity.

NVidia does everything it can to hide the fact that their devices support OpenCL. People thinks that only ancients versions of OpenCL run on NVidia devices. That is not true: 1.2 is not ancient is still as of today the main version of OpenCL used. OpenCL 1.2 is fully supported and NVidia quietly say to its large customers who refuse to use CUDA, that they will starting to support soon some OpenCL 2.0 features.

To answer your question, I am not sure either will win, but they will both exist for a long time.

Re:An Even Easier Introduction to CUDA by jupiter90000 on Saturday February 04, 2017 @04:24PM (#13569153)
Thank you, this is very helpful information I was hoping to hear.
Re:An Even Easier Introduction to CUDA by jlebar on Saturday February 04, 2017 @02:05PM (#13568431)
Honestly for those of us in machine learning, I think something like XLA will likely win over both paradigms. (Disclaimer, I work on XLA.)

https://www.tensorflow.org/versions/master/experimental/xla/

XLA much more closely matches what you want for ML than CUDA/opencl. Which isn't a surprise; it was designed specifically for ML.

Kernel launches are expensive, so any fast CUDA system has to let you compose computations into a single kernel (e.g. multiply by 5 and then take tanh). It's possible to do this in CUDA, but it requires heoric C++ template metaprogramming. It's not uncommon to have files that take ten minutes to compile. Whereas in XLA kernel fusion is nbd, because it's a JIT.

Also, because XLA is generating GPU code after it's seen your model, it can specialize computations specifically to your model. In regular TensorFlow (and I presume other ML frameworks, although I'm not at all familiar with them), you have to compile all of your kernels upfront. This means that the framework probably doesn't have the ideal set of kernels for your model, because the framework's set of kernels needs to be generic. For example, the framework probably isn't going to have a "multiply by 5 and then take tanh" kernel -- if you're lucky, it might have a "multiply by X and then take tanh", but notice that this may be slower because X is now not a constant.

In contrast, not only can XLA specialize for your weird X==5 case, but it can also specialize all of the dimensions of your arrays. This is a really big advantage in many cases.

As just one example, it's common for kernels to do something like

  int index = some computation based on threadIdx and blockIdx;
  if (index < array_len) { ... }
But in XLA we know the size of the kernel, so we know the possible values for threadIdx and blockIdx, and we know the exact value of array_len. We can therefore often optimize out the if entirely.
Re:An Even Easier Introduction to CUDA by doosra on Saturday February 04, 2017 @07:32PM (#13570023)
Justin, XLA sounds interesting. Do you assume you always have CUDA sources for ML operations in XLA? I was under the impression that closed-source libraries like cuDNN were used.

Is it possible to accurately evaluate the profitability of fusing two kernels in CUDA (effects of increased register pressure; shared memory)? On the other hand, the generic kernel and its launch parameters were probably hand tuned for performance.

Re:An Even Easier Introduction to CUDA by jlebar on Saturday February 04, 2017 @09:19PM (#13570437)
> Do you assume you always have CUDA sources for ML operations in XLA? I was under the impression that closed-source libraries like cuDNN were used.

Yes, XLA calls into cudnn and cublas. It's not a fundamental architectural thing, though; those are just the fastest matmul etc. kernels we currently have access to.

> Is it possible to accurately evaluate the profitability of fusing two kernels in CUDA (effects of increased register pressure; shared memory)?

For a human, yes, sure, just time both options. The system doesn't currently do this in an automated fashion, though. In a fashion similar to a CPU compiler's inliner, it has heuristics and makes its best guess. In general fusion is very profitable.

> On the other hand, the generic kernel and its launch parameters were probably hand tuned for performance.

Yes, and this is one of ways that XLA can lose to (say) vanilla TensorFlow today. But it's just a matter of tuning; the system is very young.

Re:An Even Easier Introduction to CUDA by disposablename on Saturday February 04, 2017 @02:19PM (#13568496)
Why no AMD GPU support?
Re:An Even Easier Introduction to CUDA by jlebar on Saturday February 04, 2017 @09:23PM (#13570445)
Why no AMD GPU support?

I think it just reflects the team's internal priorities. Patches are welcome; we want people to use this system.

It wouldn't even be tremendously hard. The XLA IR --> LLVM IR backend is relatively simple, and LLVM already has support for compiling to AMD GPUs. You'd have to split out the nvidia-isms in the generated IR. I think the biggest challenge would just be one of software engineering, namely figuring out a way to specialize the GPU backend for each of the two architectures while allowing it to share code in general.

Re:An Even Easier Introduction to CUDA by joe_the_user on Saturday February 04, 2017 @02:21PM (#13568504)
I've been evaluating Cuda and OpenCL while trying to produce some target independent code.

My impression is that while Cuda might not win, OpenCL will almost certainly lose. OpenCL seems to be a monster compromise interface which takes into account all the architectures of the members of a large consortium. It's the sort-of designed-by-committee api that a developer has to fight against to accomplish anything. Naturally its many years behind Cuda in features, etc.

An open-source library with equivalent qualities to Cuda is needed- ie, a library intended to aid developers, allow abstract c++ to be easily become parallel code, provide reasonable tools and documentation etc.

One promising example is amd's Hip

"HIP allows developers to convert CUDA code to portable C++. The same source code can be compiled to run on NVIDIA or AMD GPUs."

https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP

Re:An Even Easier Introduction to CUDA by rayuela on Saturday February 04, 2017 @03:14PM (#13568798)
Hadn't heard of this before. This is pretty cool. Does this project have official AMD support?
Re:An Even Easier Introduction to CUDA by joe_the_user on Saturday February 04, 2017 @04:36PM (#13569223)
What I remember from this being on hn months ago is that is this an official AMD project aiming to compete with nVidia.
Re:An Even Easier Introduction to CUDA by joe_the_user on Monday February 06, 2017 @01:37AM (#13577064)
Re:An Even Easier Introduction to CUDA by arcanus on Saturday February 04, 2017 @11:28AM (#13567522)
I think that the closed nature of CUDA will be its undoing. I think that a standard, like C++ amp or openMP-4.5 will be the the ultimate winner.

I liked openCL but it seems to be dying.

Re:An Even Easier Introduction to CUDA by programmarchy on Saturday February 04, 2017 @12:30PM (#13567874)
Apple seems to have abandoned OpenCL in favor of Metal, which speaks to your case of it dying.

I found Metal Compute Shaders to be very nice to work with, though. Was much easier for me to understand than OpenCL.

Re:An Even Easier Introduction to CUDA by arcanus on Saturday February 04, 2017 @12:39PM (#13567911)
I also like metal, but it is not yet performant for high performance computing, which is more my wheelhouse. I'm also skeptical it will be popular if it does not get picked up by the GPGPU folks, but time will tell.
Re:An Even Easier Introduction to CUDA by ahelwer on Saturday February 04, 2017 @11:31AM (#13567536)
Is C++ AMP still going? I learned it about five years ago back when I was involved in GPU programming, haven't heard a thing about it since then.
Re:An Even Easier Introduction to CUDA by jupiter90000 on Saturday February 04, 2017 @11:50AM (#13567640)
Looks like it might be dead/dying:

http://stackoverflow.com/questions/34969287/what-is-the-curr...

Although perhaps would still be useful in its stale state, not sure.

Re:An Even Easier Introduction to CUDA by jupiter90000 on Saturday February 04, 2017 @11:30AM (#13567533)
I'm not familiar with those other standards, thanks for mentioning them. I'll check them out.