From my experience with CUDA/OpenCL, its easy to get a working algorithm that provides some speedup, but getting it to fully utilize the GPU takes some manual optimization.

Things like optimizing memory accesses to prevent bank conflicts, using per thread-group shared memory effectively to reduce global memory accesses are a bit tricky to get right, and I'm not sure how well auto-generated code would be able to do this.