not sure if this is what you are looking for but I had success with this Java library http://aparapi.com/, it looks at function's JVM byte code and runs it on GPU if it can, otherwise falls back to CPU. I suspect same thing can be done in other languages.

https://github.com/Syncleus/aparapi/

"Aparapi allows developers to write native Java code capable of being executed directly on a graphics card GPU by converting Java byte code to an OpenCL kernel dynamically at runtime. Because it is backed by OpenCL Aparapi is compatible with all OpenCL compatible Graphics Cards."