PyTorch – Tensors and Dynamic neural networks in Python

Re:PyTorch – Tensors and Dynamic neural networks in Python by vegabook on Wednesday January 18, 2017 @01:41PM (#13429019)

Guess there's no escaping Python. I had hoped Lua(jit) might emerge as a scientific programming alternative but with Torch now throwing its hat into the Python ring I sense a monoculture in the making. Bit of a shame really because Lua is a nice language and was an interesting alternative.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by jjawssd on Wednesday January 18, 2017 @01:50PM (#13429114)

Lua is extremely flexible to the point where there is basically no standard library. This causes problems with code reuse and moving between codebases because everyone does things drastically differently. Compare this to Numpy in the Python world, a single fundamental package for scientific computing in Python.

Lua is less used than Python in the scientific community, and a lot of the most innovative machine learning researchers already work with C++ and Python. Using yet another language with only marginal benefit increases cognitive load and drains from the researcher's mental innovation budget, forcing the researcher to learn the ins and outs of Lua rather than working on innovative machine learning solutions.

Lua is a nice language. Python 3 is a nice language and there are many new exciting features and development styles (hello async programming?) in the making which will prevent a monoculture from forming in the near term.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by vegabook on Wednesday January 18, 2017 @01:54PM (#13429154)

Thanks for the interesting and informative comment. Do I sense just a tiny bit of regret though? Yet another Python interface. YAPI. You heard it here first. And no, Py3 is not that nice. Too much cruft by far. And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data. Arguably luajit obviates the need for C , something you can't say about Python. Disclosure: I am a massive, but increasingly disenchanted, user of Python. I had actually started looking at Torch7, foregoing tensorflow, precisely because of Lua. But the walls are closing in....

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by jjawssd on Wednesday January 18, 2017 @02:05PM (#13429269)

A very large portion of performance problems can be mitigated with the use of cython and the new asyncio stuff.

asyncio success story: https://magic.io/blog/asyncpg-1m-rows-from-postgres-to-pytho...

cython: http://scikit-learn.org/stable/developers/performance.html

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by vegabook on Wednesday January 18, 2017 @02:16PM (#13429373)

Luajit is at least 10x faster than python and easily obviates the need to mess around with cython. That's an easy win for Lua. Let's be honest: Torch has decided that if you cannot beat them, join them. It is about network effects. Not about Python better than Lua intrinsically.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by baq on Wednesday January 18, 2017 @05:02PM (#13430741)

but why do you care if luajit is faster than python if everything that matters is computed on the GPU anyways?

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by jjawssd on Wednesday January 18, 2017 @02:22PM (#13429432)

Can't argue with that

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by inlineint on Thursday January 19, 2017 @06:33AM (#13434951)

An alternative to Cython is Numba [1], which speeds up some cycles in pure Python by just adding a single decorator.

[1] http://numba.pydata.org/

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by cr0sh on Wednesday January 18, 2017 @02:17PM (#13429390)

> And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data.

Then use Lua for that, if you are more comfortable there and want/need the speed bump. There's nothing that says an entire project or whatnot has to be developed in a singular language.

Use each tool to its strengths, as your needs, requirements, and abilities dictate.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by vegabook on Wednesday January 18, 2017 @02:23PM (#13429453)

There always has to be someone rolling out the horses-for-courses pitch. No. I wanted Lua to gain traction with other people. That's the point. I would have liked the Lua sci-ecosystem to be healthy as an alternative.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by infinite8s on Wednesday January 18, 2017 @04:34PM (#13430487)

Is there an equivalent to numpy in the Lua space?

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by apaszke on Wednesday January 18, 2017 @07:03PM (#13431708)

torch?

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by webmaven on Thursday January 19, 2017 @08:12PM (#13440978)

> And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data.

Is that true even if the Python used is PyPy rather than CPython?

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by etiene on Friday January 20, 2017 @05:36AM (#13442799)

I like Lua more than I like Python and all of this makes me sad. I wished more people were putting their hearts into getting the Lua's ecosystem going instead of into things like this.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by argonaut on Wednesday January 18, 2017 @02:59PM (#13429750)

You're making the language out to be more important than it really is.

The Python that you write when using these frameworks just the glue code / scripts. All you're doing is calling the framework's functions. Most of it gets thrown away (as researchers). The stuff that doesn't is self-contained and usually short. You're not writing 100k+ line codebases.

Lua may be faster for certain tasks (data processing), but the time it takes for does tasks is usually a rounding error in deep learning. Not to mention you can still code in C/C++ with pytorch.

If there is a monoculture in machine learning, it would be the deep learning monoculture.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by statsmatscats on Wednesday January 18, 2017 @03:23PM (#13429960)

Here is an emerging julia alternative http://www.breloff.com/transformations/

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by eva1984 on Wednesday January 18, 2017 @07:25PM (#13431843)

No, there is already no escaping from CUDA, so it is already a monoculture nevertheless.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by crudbug on Wednesday January 18, 2017 @02:50PM (#13429678)

I was also on this bandwagon. Its about pythonic syntax not semantics that drives this.

If only Mike Pall created a transpiler infrastructure layer on top of LuaJIT.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by wodenokoto on Wednesday January 18, 2017 @02:14PM (#13429354)

There's also R and Julia and there are still plenty of people building neural networks in C.

[ Parent | on HN ]

Re:PyTorch – Tensors and Dynamic neural networks in Python by attractivechaos on Wednesday January 18, 2017 @05:49PM (#13431173)

It is easy to build a multi-layer perceptron purely in C. You can roll your own or use a library like FANN. However, so far as I know, very few (darknet is the only example I know of) are using C to build a bit more complex networks like CNN/RNN, let alone those topologically complex networks in research domain.

[ Parent | on HN ]