NumbaPro is a python library that let's you JIT python into CUDA or CPU vector extensions via decorators.
OpenACC is similar (or possibly developed out of) OpenMP in that it takes a directive driven approach to accelerating code. Given a loop you can add a pragma above it with instructions of how it should run a GPU and it will make it happen.
[1] http://docs.continuum.io/numbapro/ [2] http://www.openacc-standard.org/
EDIT: I forgot to mention that I believe we'll be seeing less writing of GPU code directly and more using of libraries that do the hard work for you. Some algorithms on GPUs are quite tricky to get right, and even with a language that makes them easy to express they are non-trivial to design and test. (Nvidia already provides Thrust with cuda which provides a nice STL like library that implements a lot of common routines)
Structured data and nonuniform computations are very hard to express right now on the GPU and typically require a long and terrible process of "creative" CUDA programming. Harlan seems to be pushing the boundaries of what sorts of GPU programs we can effectively/efficiently implement. But, because the syntax is unusual, you're saying it's irrelevant compared to NumbaPro...which is what, a thin wrapper over elementwise array operations?
What I believe GPU programming needs is tools that people can easily add to their current environments or that they can easily extend their current algorithms with. NumbaPro, OpenACC, and Thrust allow that.
In addition, the reason it's hard to express structured data and nonuniform computation is because those aren't things GPU architecture excels at. It excels at doing uniform operations over large chunks of memory. I'm sure that Harlan is pushing the boundary of what you can do on a GPU, but it's not going to exceed the limitations imposed by the hardware.
If you write simple GPU kernels (like those that Theano/Thrust/Copperhead/NumbaPro let you easily express) then you're mostly stuck "doing uniform operations over large chunks of memory". However, the latest GPUs are packed with features for going beyond this simple model. There's a rich set of global atomic operations, a fast register shuffle, better caching and most importantly: dynamic parallelism via nested kernel invocations. We're not programming for the G80 any more, Keplers can run a much larger swath of programs. Sure, you won't reach the theoretical peak FLOPS by traversing irregular structures via recursive kernels but you might still beat the pants off a CPU.
...though I am concerned that abstracting this stuff too much makes it less obvious how to optimize for the GPU. This may be a language for people that already have a solid grasp on GPU programming.
Um. OpenCL/Cuda look absolutely nothing like that.
Absolutely right. You can also include an external C file from C++ [1]; that makes it so much prettier.
[1] http://developer.amd.com/tools-and-sdks/heterogeneous-comput...
Happily, writing this answer I discovered that OS X 10.7+ ships with OpenCL already! Can't believe I missed that last time.
For anyone else wanting to try it, the first example is pretty straight forward:
https://developer.apple.com/library/mac/#documentation/Perfo...
(You'll also want the code examples at https://developer.apple.com/library/mac/#documentation/Perfo... )
That, coupled with the wretched GPU options available on the Mac Pro, make the current stack not really cost effective. (I think a 5770 is still $250.)
However, it is better than CPU-only.
I've found slaving a cheap Linux box with expensive GPUs to be a much more gratifying experience.
They both run on the same hardware, but I don't have enough (or really any) experience with OpenCL to say if it feels more or less powerful than GLSL.
Edit: It turns out that I have no OpenCL hardware support (mid 2011 MBA), so I guess the hardware isn't necessarily completely similar (i.e. it can support GLSL and not OpenCL)