News for hackers. Stuff that matters.
Re:Neural Networks in JavaScript by sarreph on Monday May 09, 2016 @06:23AM (#11658695)
From the article:

> Within the last years, multiple Javascript frameworks were developed that can help you to create, train and use Neural Networks for different purposes.

Due to the popular discourse on how 'unsuitable' JS is for anything but front-end development°, does anybody have any insight on how the efficiency of JS neural net libraries compares to that of the classic Python et al. tools? I've been trying to get started in neural nets for a while now, but am curious as to if learning it via JS will be a limiting factor that I will ultimately have to switch to a more efficient/suited platform.

° Something that I do not agree with myself, as a Node.js and back-end JS developer

Re:Neural Networks in JavaScript by poorman on Monday May 09, 2016 @09:46AM (#11659769)
I wrote the the original machine learning for our company using a fork of brain.js. We launched to 300k+ users the first day with a model training time of under 10 seconds. I've since re-written our machine learning in Go. For getting something out the door quickly, Javascript is great.

It worked well for us because we had to stream the data into the network for training, --all the training data did not fit into RAM on Heroku's 1G dyno. Node.js does this IO transfer really well. (Full disclosure, I wrote the streaming implementation for brain.js.)

Also, training artificial neural networks is a serial computation for the most part. (Lots of sigmas). There are parallel optimizations you can make but probably won't see the benefits (in terms of training speed) until you have high dimensional data. Doing this sort of thing on a GPU is also orders of magnitude faster.

One thing to note however, if you plan on serving a high number predictions from trained models in Javascript, you will see blocking because each prediction does require a small bit of computation and Javascript / Node.js is single threaded.

Re:Neural Networks in JavaScript by fapjacks on Monday May 09, 2016 @03:45PM (#11662562)
You wouldn't happen to have written about your adventures, would you?
Re:Neural Networks in JavaScript by poorman on Tuesday May 10, 2016 @02:45PM (#11669550)
No, I haven't written anything.
Re:Neural Networks in JavaScript by binarymax on Monday May 09, 2016 @07:57AM (#11659074)
JS is mostly 'unsuitable' because of the lack of integration into GPGPU.

This is one of the reasons I was hoping WebCL[1] (now dead[2]) would take off. It would have enabled GPGPU work that is pretty much a prerequisite for anything that would be taken seriously by the community.

There was talk of ARB_compute_shader support as mentioned in the second ref link, but I haven't seen anything else in that area.

[1] https://www.khronos.org/webcl/

[2]https://bugzilla.mozilla.org/show_bug.cgi?id=664147#c30

Re:Neural Networks in JavaScript by onion2k on Monday May 09, 2016 @10:58AM (#11660278)
JS is mostly 'unsuitable' because of the lack of integration into GPGPU.

Does that mean that using an already trained model would be slow? If I wrote something to train a model on a GPU, could I then export the data to use with JS in a browser?

Re:Neural Networks in JavaScript by steplee on Monday May 09, 2016 @04:36PM (#11662934)
Training is essentially running the net forward then backward on many training examples. Using the model is running it forward on one input, so it would be much faster depending on your specific model and use cases
Re:Neural Networks in JavaScript by netheril96 on Monday May 09, 2016 @06:37AM (#11658749)
For any non trivial deep network, any CPU based training is horrendously slow, even if multithreaded. So, Javascript is doomed at it. Python is not, as it can interact with native binaries which may talk with the GPU.

> Something that I do not agree with myself, as a Node.js and back-end JS developer

NodeJS mostly works when CPU time is not the limiting factor, that is, powering the Web. Neural network is the polar opposite.

Re:Neural Networks in JavaScript by lucideer on Monday May 09, 2016 @06:49AM (#11658797)
> Python is not, as it can interact with native binaries

It sounds like what you actually mean here is that Python is equally as unsuitable and must instead use native binaries. Which one could just as easily do with JS...

A reasonable argument could be that such native binaries are available in the Python community and already in use as stable existing solutions, but that's more about ecosystem than language suitability.

Additionally, while I'm not too sure what convnet, node-mind, brain.js, etc. referenced in the article are using internally, there are gpu-oriented solutions being written in JS like https://www.npmjs.com/package/weblas

Re:Neural Networks in JavaScript by adrianN on Monday May 09, 2016 @09:59AM (#11659859)
Python is dog slow. This is why it is used as a glue language for C modules. This is the canonical way to do things. Javascript OTOH is decently fast on its own, but you can't really interact with libraries written in C or FORTRAN, as far as I know.
Re:Neural Networks in JavaScript by notduncansmith on Monday May 09, 2016 @10:50AM (#11660237)
Node has been able to interoperate with C/C for some time, via node-gyp and node-ffi.
Re:Neural Networks in JavaScript by 15155 on Monday May 09, 2016 @10:45AM (#11660193)
Re:Neural Networks in JavaScript by adrianN on Monday May 09, 2016 @10:48AM (#11660212)
Cool, good to know.
Re:Neural Networks in JavaScript by amelius on Monday May 09, 2016 @06:41AM (#11658770)
Well, somebody could write a NodeJS library that talks to the GPU.
Re:Neural Networks in JavaScript by brudgers on Monday May 09, 2016 @09:10AM (#11659515)
Atwood's Law makes it inevitable that someone will if no one has already.
Re:Neural Networks in JavaScript by pjonesdotca on Monday May 09, 2016 @09:35AM (#11659693)
Re:Neural Networks in JavaScript by amelius on Monday May 09, 2016 @10:14AM (#11659971)
Re:Neural Networks in JavaScript by spraak on Monday May 09, 2016 @06:47AM (#11658790)
I think they mean that JS is good for more than just FE which doesn't necessarily mean Neural Networks..
Re:Neural Networks in JavaScript by nl on Monday May 09, 2016 @08:01AM (#11659096)
The current frameworks are deeply (ha!) unsuitable for any serious work, by which I mean anything where a neural network would approach outperform other methods.

However there is some possibility that technologies like WebCL and replacements will overcome this limitation by allowing access to GPU resources. Further reading: [1][2][3]

[1] https://github.com/waylonflinn/weblas

[2] https://github.com/karpathy/convnetjs/issues/13

[3] https://github.com/mikeseven/node-webcl

[4] https://github.com/stormcolor/webclgl

Re:Neural Networks in JavaScript by dancsi on Monday May 09, 2016 @06:39AM (#11658765)
I would guess that it is very unsuitable, since for anything but the most trivial models, even those 'Python' libraries resort to highly-optimized (compiled) C subroutines, especially for the training part. Even then you require a highly nontrivial amount of time to train the network, especially if you are working with medium to large datasets.

Later on, when the speed is not so critical, you can do inference with those pretrained weights and your model implemented in Javascript.

Re:Neural Networks in JavaScript by wodenokoto on Monday May 09, 2016 @07:10AM (#11658875)
With javascript you have ASM.js which should get you close to native C speed for exactly these types of operations.

But I don't know how well you can talk to the GPU

Re:Neural Networks in JavaScript by tuananh on Monday May 09, 2016 @10:10AM (#11659953)
not as close as you think it is. perhaps C is still 2x, 3x faster.
Re:Neural Networks in JavaScript by imaginenore on Monday May 09, 2016 @07:58AM (#11659080)
Most Python neural network libraries actually have the critical parts written in C++, and use GPUs. JavaScript has asm.js, which isn't as fast. There's also NaCl in some browsers.

What JS really needs is WebCL. That will revolutionize JS capabilities.

Re:Neural Networks in JavaScript by lisivka on Monday May 09, 2016 @03:53PM (#11662615)
What we need is support for optimized/hw accelerated versions of JS libraries in browser.

Eample:

A page declares that it will use library foo, e.g. <lib name="foo" version="^1.0"/>. Browser, e.g. firefox, looks into common repository and sees that there is two versions of "foo": one generic, and one optimized for firefox-x86_64, with native code, so it used optimized version instead.

Of course, code must come from trusted source and be signed, to reduce risks.

Re:Neural Networks in JavaScript by alimw on Monday May 09, 2016 @07:21AM (#11658924)
A corollary of Parkinson's Law is that, whatever language you start with, your neural net project will expand to fill the processing power available to it.