News for hackers. Stuff that matters.

hannes-brt

Re:GPU-Accelerated Deep Learning Library in Python on Friday December 06, 2013 @01:45PM (#6862770)
What kind of benchmarks were you looking for? Benchmarks with respect to speed or MNIST results? For MNIST, using the DNN example, Hebel gets results that are in line with reported results on the MNIST homepage [1] for similar models (pure Neural nets with cross-entropy error and no distortions etc), which is about 1.3% error.

There are some unit tests in Hebel though [2], even though coverage is not complete.

[1] http://yann.lecun.com/exdb/mnist/ [2] https://github.com/hannes-brt/hebel/blob/master/hebel_test.p...

Re:GPU-Accelerated Deep Learning Library in Python on Friday December 06, 2013 @01:40PM (#6862745)
That would be possible, but since Hebel is mainly meant to be used in research I don't think it's a big priority now. The most important reason to do this would be to allow development on laptops and workstations without NVIDIA cards and to run the finished model on CUDA hardware later.

As far as embedded devices go (I assume you're talking about ARM cpus etc), they are probably too underpowered to run Neural nets anyway, or models would have to be written in highly specialized C.

Re:GPU-Accelerated Deep Learning Library in Python on Friday December 06, 2013 @01:36PM (#6862702)
The reason why haven't heard of it is probably because I only put the code on Github less than two weeks ago ;) - I've really been blown away by the response though.
Re:GPU-Accelerated Deep Learning Library in Python on Friday December 06, 2013 @01:35PM (#6862688)
This is awesome and I'm sure it will be a boon to app developers who want to include Machine Learning capabilities in their apps.

It looks to me as though Ersatz's focus is on providing a limited range of relatively standard models, but make them highly accessible, stable, fast, and suitable for production, whereas most available frameworks like Theano, PyLearn2, etc are more geared to the tinkering researchers and less to be used in actual products.

Re:GPU-Accelerated Deep Learning Library in Python on Thursday December 05, 2013 @02:15PM (#6856471)
Yes, you can't come up with a hard number, but a 10x speedup is really the baseline. 50x and more are not out of the question.
Re:GPU-Accelerated Deep Learning Library in Python on Thursday December 05, 2013 @02:14PM (#6856459)
Yes, this is true. Theano is just a GPU-computing framework that you can use to implement any kind of numerical model. Hebel implements some specific models, but it uses PyCUDA as a backend.
Re:GPU-Accelerated Deep Learning Library in Python on Thursday December 05, 2013 @02:13PM (#6856446)
Yes, I'm sorry about that, but it's not going to change. I personally don't have experience developing for OpenCL and the deep learning community seems to very much have embraced CUDA.
Re:GPU-Accelerated Deep Learning Library in Python on Thursday December 05, 2013 @02:10PM (#6856429)
Yes, Hebel doesn't have a ton of features and a kitchen-sink of different models yet, but I hope that's going to change. There are lots of things that are quite easy to implement in the current framework, such as: - Neural net regression - Autoencoders - Restricted Boltzman machines

There's a lot of interest for convolutional networks and the best way to implement it will be to wrap Alex Krizhevsky's cuda-convnet, like DeepNet and PyLearn2 have, but this will require a bit more effort.

With respect to other deep learning packages, Hebel doesn't necessarily do everything differently, but depending on your needs it may be the best choice for a particular job.

PyLearn2 is big and monumental and although I haven't used it much personally, it seems to excellent. But as you mentioned, it's not necessarily easy to use and if you want to extend it, you have to learn the Theano development model, which takes some time to grok.

DeepNet is quite similar to Hebel in its approach (even though it offers more models right now). However, DeepNet is based on cudamat and gnumpy, which I have found to often be quite unstable and slow. Hebel is based on PyCUDA which is very stable and according to some preliminary tests I did runs about twice as fast as cudamat.

So, the idea of Hebel is that it should make it easy to train the most important deep learning models without much setup or having to write much code. It is also supposed to make it easy to implement new models through a modular design that lets you subclass existing layers or models to implement variations of them.

Re:GPU-Accelerated Deep Learning Library in Python on Thursday December 05, 2013 @01:55PM (#6856324)
Hello Hacker News, this is the Hebel developer. Thanks for the attention and the Github stars!

I'll do my best to answer any questions in the thread. Looking forward to you trying out Hebel and your pull requests ;)