Torch7 – Scientific computing for LuaJIT

Re:Torch7 – Scientific computing for LuaJIT by smhx on Sunday June 22, 2014 @05:07PM (#7929216)

I am one of the maintainers. From information first-hand, Torch is used by:

- Facebook

- Google DeepMind and slowly Google Brain is moving as well.

- Certain people at IBM

- NYU

- IDIAP

- LISA LAB (not exclusively, but some students started using it)

- Purdue e-lab

- Several smaller companies (10-100 companies)

There will definitely be several commonly asked questions, and this is my personal perspective on them.

Why torch/lua, why not python+?*

No reason. Just because. Mostly because LuaJIT is awesome (with it's quirks) and LuaJIT is extremely portable. (we embed torch routinely in tiny devices, afaik not practically possible with python).

Is Torch better than Theano/etc.etc.?

Better and worse. Every framework has it’s oddities. I like the super-simple design and the compactness of traversing from high-level easy-to-use API to bare-metal C/assembly.

Also, torch’s ecosystem was grown not with exclusively lab experiments in mind, with Yann’s strong robotics research, packages were developed with practicality in mind all the time. Custom chips are being developed for convnets (TeraDeep) and they use Torch.

Where’s the doxx???

If there’s documentation, I’ve tried to make people aware of it, mostly by consolidating everything torch to this one page: https://github.com/torch/torch7/wiki/Cheatsheet

What about Julia?

I like Julia a lot, and it’s definitely cool, but the packages for NN and GPUs aren’t very strong, so advantages over Julia are simply code that’s already written.

If there are any more questions, feel free to ask them here or just open an issue on the github package.

Thanks for reading.

Edit: Apologize for the formatting, not very good at hackernews

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by benanne on Sunday June 22, 2014 @09:43PM (#7929803)

I'd love to try out Torch7, but I've grown accustomed to Theano's automatic differentation, which is the killer feature for me. That's the one thing keeping me from exploring the alternatives. Not having to calculate any gradients makes trying different architectures and activation functions a lot less daunting.

Are there any plans for Torch7 to support something similar? I suppose it would be nontrivial since to my knowledge Torch7 does not use symbolic representations of the computations internally, which is what enables Theano to do automatic differentation in the first place.

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by smhx on Sunday June 22, 2014 @10:12PM (#7929870)

I'm writing a calculus package that's inspired by Julia's, which would have an automatic differentiation engine, but it is nowhere close to being finished, nor is it a priority at this point. So sadly, we wont see Sander using torch anytime soon :)

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by agibsonccc on Sunday June 22, 2014 @10:56PM (#7929984)

Torch is an amazing project and has been a great inspiration for my leap in to deep learning. I wish the maintainers the best of luck. It's ambitious to write a deep learning lib. That being said, you guys are doing a great job.

The comparison helps quite a bit. Mind if I lift this for our site? I'd link back to this thread as well.

Thanks!

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by smhx on Sunday June 22, 2014 @11:07PM (#7930006)

go for it.

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by ksec on Sunday June 22, 2014 @11:12PM (#7930022)

*Mostly because LuaJIT is awesome (with it's quirks)

Could you give some example of its quirks?

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by smhx on Sunday June 22, 2014 @11:28PM (#7930075)

my biggest annoyance with LuaJIT is the lack of 64-bit addressing, and there's no easy fix for this. Mike Pall said he'd fix it in LuaJIT 3.0 (with an overhaul of the GC), whenever that'll be.

http://hacksoflife.blogspot.com/2012/12/integrating-luajit-w...

Edit: to clarify, this does not mean torch cant use more than 2GB, but native lua allocations cant be more than 2GB, so ffi-based or C based allocations (which is how torch tensors are allocated) have no such limits.

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by neomantra on Monday June 23, 2014 @09:08AM (#7931695)

This is a big annoyance for me as well. I use jemalloc to make sure that my FFI allocations don't 'steal' this lower memory from my pure Lua allocations. It works great on Linux, but only so-so on OSX.

http://www.canonware.com/pipermail/jemalloc-discuss/2014-Feb...

https://gist.github.com/neomantra/9122165

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by pavanky on Sunday June 22, 2014 @05:32PM (#7929301)

Hi,

I am the lead engineer at ArrayFire[1]. Is there anyway I can get in touch with you ?

[1]http://www.arrayfire.com/docs/index.htm

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by smhx on Sunday June 22, 2014 @10:17PM (#7929882)

Sure, the best way to get attention is to open an issue on github.com/torch/torch7 (that way you get the attention of all the core developers). If you'd like to ask something more discrete, I'd suggest you email one of the four here: https://github.com/orgs/torch/members

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by ludamad on Sunday June 22, 2014 @10:00PM (#7929842)

It may seem obvious, but I feel like the page you linked could use an occurrence of the word 'C++'. The code snippet was obvious to me, but just a thought.

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by pavanky on Sunday June 22, 2014 @10:39PM (#7929941)

Thanks for the feedback! While the code snippet is from C++, we have language wrappers for Java, R and Fortran. We'll try to be more clear about this in our documentation.

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by psandersen on Sunday June 22, 2014 @06:39PM (#7929461)

ArrayFire looks very interesting, as a novice a drop in enhancement of the nn (or nnx for dropout) package for torch would be very useful.

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by pavanky on Sunday June 22, 2014 @10:32PM (#7929920)

We are getting there. In the mean time we have an example on how you can go about building one yourself. You can see it here: https://github.com/arrayfire/arrayfire_examples/blob/master/...

[ Parent | on HN ]

Re:Torch7 – Scientific computing for LuaJIT by psandersen on Wednesday June 25, 2014 @08:57PM (#7947604)

Thanks for the response, currently I'm doing R&D with easier to use high level representations and if it proves fruitful will be looking at getting the most cost effective speedups in about 6 months.

Whats the license for the free edition? I would be sticking to open source code until we secure capital, but would be very interested in your commercial services down the line.

[ Parent | on HN ]