This is made in a way that it compiles automatically into an op for Theano or TensorFlow (PyTorch could easily be added as well) and for both CPU and CUDA/GPU.
Re:Unifying the CUDA Python Ecosystem
by dwrodri on Friday April 16, 2021 @01:28PM
(#26836726)
I don't know specifics about Copperhead in particular, but Bryan Catanzaro (creator of Copperhead) is now the VP of Applied Deep Learning Research at Nvidia. He gave a talk at GTC this year, which is how I heard about all of this in the first place.
I think I found it here: https://github.com/bryancatanzaro/copperhead
But I'm not sure what the state is. Looks dead (last commit 8 years ago). Probably just a proof of concept. But why hasn't this been continued?
Blog post and example: https://developer.nvidia.com/blog/copperhead-data-parallel-p... https://github.com/bryancatanzaro/copperhead/blob/master/sam...
Btw, for compiling on-the-fly from a string, I made something similar for our RETURNN project. Example for LSTM: https://github.com/rwth-i6/returnn/blob/a5eaa4ab1bfd5f157628...
This is made in a way that it compiles automatically into an op for Theano or TensorFlow (PyTorch could easily be added as well) and for both CPU and CUDA/GPU.