If you look at the profiler screenshot at the bottom of the page, the time spent in the cuRAND kernel (cyan) is ~2x longer than in the numbapro kernel (purple). Even if the numbapro kernel is written in CUDA-C and suppose it will be a lot faster, you still won't hit 100x speedup. In addition, all kernels are double-precision, the 100x speedup is more common for single-precision kernels.

I'm be surprised if there was that large of a speedup going from a well-written numpy implementation to C/C++ (assuming a non-parallel implementation in both)...

In my experience, it's usually more like 2x-5x for a straight translation of the algorithm from numpy to C. (Though I have to confess that I often write rather naive C...)

Numpy performs quite well if you avoid a few common pitfalls. It certainly can be beat, but it's often faster than people think it is.

That matches my experience as well. I've never gotten more than a 3x speed up by doing a straight numpy->C rewrite on actual real world code (actually doing one right now, and I've gotten a 2x speed up after my first pass).

NumbaPro generates CUDA code that runs on the GPU. I don't see how using C++ to do it would necessarily make it faster as the code executing on the CPU end of things is not the bottleneck.

From my experience with CUDA/OpenCL, its easy to get a working algorithm that provides some speedup, but getting it to fully utilize the GPU takes some manual optimization.

Things like optimizing memory accesses to prevent bank conflicts, using per thread-group shared memory effectively to reduce global memory accesses are a bit tricky to get right, and I'm not sure how well auto-generated code would be able to do this.