Dumb question, but can someone give me a summary of how you can implement this in webgl?

I thought there are only vertex/fragment shaders and compute shaders aren't supported yet? Do you just pretend everything is pixel data?

Author of deeplearn.js here. A quick summary:

We store NDArrays as floating point WebGLTextures (in rgba channels). Mathematical operations are defined as fragment shaders that operate on WebGLTextures and produce new WebGLTextures.

The fragment shaders we write operate in the context of a single output value of our result NDArray, which gets parallelized by the WebGL stack. This is how we get the performance that we do.

Which... is pretty much how GPGPU started in the early 2000. Sad/funny how we go through this cycle again.

It will be interesting to see if the industry will produce a standard for GPGPU in the browser. Giving that the desktop standard is less common than a proprietary standard.

This is still done in pretty much every game engine I've worked with (for general computation used to support rendering as much as the rendering itself). It's frankly extremely practical and better than many GPGPU apis because it matches what the hardware is doing internally better (GPU core warps, texel caches, vertex caches, etc).

> It will be interesting to see if the industry will produce a standard for GPGPU in the browser.

They did: webcl Sadly, it had multiple security issues so the browsers that had implemented it in their beta channels (just Chrome and Firefox, I believe) ended up removing it. And now, I think it's totally stalled and no one is planning on implementing it.

Also sadly, SIMD.js support is coming along extremely slowly.

WebGPU conversations are ongoing: https://en.wikipedia.org/wiki/WebGPU

WebAssembly is coming along quite nicely.

And SwiftShader is a quite nice fallback for blacklisted GPUs. They simulate WebGL on the CPU and take advantage of SIMD: https://github.com/google/swiftshader

Are there plans to offer the whole zoo?

http://www.asimovinstitute.org/neural-network-zoo/

As I understand, deeplearn.js is more of a kitchen than a prepared meal. Part of the library is referred to as “numpy for the web” with classes to run linear algebra equations efficiently, leveraging the GPU. I don’t see why you couldn’t use those pieces to set up other networks. I think the name “deeplearn.js” is moreso capitalizing on the branding momentum of “deep learning” rather than being the demonstration of one kind of network. I’m in the middle of introductory machine learning classes, so I hope someone will correct me if I’m wrong.

You're right. Some history:

We wanted to do hardware accelerated deep learning on the web, but we realized there was no NumPy equivalence. Our linear algebra layer has now matured to a place where we can start building a more functional automatic differentiation layer. We're going to completely remove the Graph in favor of a much simpler API by end of January.

Once that happens, we'll continue to build higher level abstractions that folks are familiar with: layers, networks, etc.

We really started from nothing, but we're getting there :)

Thanks for the explanation! I recently have been working on my own deep learning library (for fun) and was doing something similar. Aren't GL textures sampled with floating point units inexactly? Do you just rely on floating point error to be small enough that you can reliably index weights?

I ended up switching to OpenCL since I am running this on my desktop. Just curious to see what you did. Thanks!

You can set nearest neighbor interpolation for the texture (aka no interpolation), and gl_FragCoord can be used to determine which pixel the fragment shader is operating on.

Sick! That is a world-class hack

It's not really a hack, it's just using the GPU's parallel computing capabilities to compute things in parallel. This technique has been around for ages.

Languages buddy, languages.. As much as languages were a barrier for human culture to spread their ideas, it's analogous in the computing world.. JS is catching up with many concepts that were prevalent in other languages/environments. Also due to JS it is now becoming more accessible and popular to the commoners..

Compute shaders are not supported in webgl, but is possible to perform vector operations by rendering to a texture with the fragment shader. It's basically a hack. The trick is rendering an image without putting it on the screen, storing arbitrary data into the pixels. This has limitations, but is actually good enough for many vector and matrix operations. I believe this method was even used with desktop opengl when gpus were first being used for general computing and didn't have more flexible apis yet like opencl/cuda

> Do you just pretend everything is pixel data?

Pretty much.

It's inconvenient because shaders are meant for vertices/fragments but it still works.

It's super sad to me, that the only convenient cross platform way to do deep learning is a hack on top of JS.

Not sad - this is the beauty of modern JS.

This is just the beginning :)

<3

Folks are downvoting this, but they must clearly have drowned themselves in kool-aid.

You want fast matrix multiplication? Pretend you're doing texture shading in a language primarily used for dynamic web frontends!

Why pretend? A texture is just an array of floats. The thing preventing you, the programmer, from seeing it as something beyond an "image" is really yourself. In modern graphics, we use textures to represent all kinds of data. Projections onto different spherical harmonic modes, luminance histograms, atmospheric data, etc.

A texture is just data and a shader mutates that in a highly parallel way with specific coherence semantics. Before knocking it, I suggest educating yourself a bit more.

What are we pretending here? A GPU is designed to do multiplication. We're doing multiplication. Calling it "texture" or "shading" or whatever else is only because of the GPU's root in gaming, otherwise the problems are exactly the same.

Yeah I don't see how I stepped on any toes. Deep learning libraries have always been a pain in the ass to install. And only support some OSes and GPUs. There's nothing wrong with JS or webgl, but it seems an incredibly inelegant choice for this.

pip install tensorflow is pretty darn easy....

Granted it doesn't have AMD support, I'm fairly confident even the CPUs can out perform the javascript library.

I wasn't aware it worked at all on Windows now, that's cool. Doesn't seem very simple to install though. This is the installation guide for just one of the prerequisites: http://docs.nvidia.com/cuda/cuda-installation-guide-microsof... And as you mention, only Nvidia.

Cuda and cuDNN; good times :) Not cross platform, but nvidia does supply a bunch of packages now for different OS's to get that stuff installed.

When you want performance the only way is getting close to the hardware. That is why deep learning is usually linux+nvidia. That is why game engines use/used C or even assemble code for critical parts.

Don't expect anything better soon as companies focus on performance per watt and will only develop things in that path.

Or you improve your algorithms and use the existing hardware (think distributed computing on cheap HDDs).

Companies will improve the algorithms and go low level with hardware instructions. The problem with GPUs is that they have special instructions for some operations. Not using them can be 20 times slower. You are not going to improve the algorithms 20 times, ND if you do you can still improve it another 20 times with hardware instructions. You do both if you can. That is a big saving in electricity in data centers.

I'm not sure how convenient this will be for most use cases.

JavaScript in the browser lacks good data manipulation libraries (nothing that comes close to using Pandas, or R). Even loading/saving files with the browser is a PITA (by design really, for security reasons).

I can see a day when JavaScript (or web assembly) has APIs like Numpy and Pandas, but for now It's simply easier to install one of those than try and do everything in JavaScript.

Presumably this will usually be run from a JS engine like Node independent of a browser. Which will have all the libraries needed for loading data and such. In theory it will be an order of magnitude faster than numpy because it uses the GPU (I don't know what the actual performance is though.) Certainly the JS will be much faster than python or R.

Deeplearn.js is designed for webGL on a browser. I'm not sure if it'll work in Node. And on a server you'd probably want to target OpenCL.

Node adds file support, but libraries for stuff like dataframes are still in their early days. That might not be a problem for example problems, but real world data needs cleaning up before you can start doing ML.

It doesn't work in node yet, a relevant issue: https://github.com/PAIR-code/deeplearnjs/issues/234

I've been waiting years for this. Once compute shaders became available to the browser I planned to finally get around to writing some code because you could project out to so many platforms. This sucks the entropy out of the coding task. You don't have to have multiple implementations, with unique bugs on multiple platforms.

JS has improved over the year but you can also go with a typed language if you wanted like purescript or typescript.

I guess I'll have to keep waiting, this isn't using compute shaders apparently.

Why wait?

I guess I targeted compute shaders in the browser as a good time to revisit linear algebra and ANN since I could expect improved performance, improved programming model and improved portability.

You have created an abstraction that is pretty portable. You'll probably be able to capture new performance enhancements as they occur on web runtimes. Maybe I'll try it out.

What's so hard about `pip install tensorflow`?

pip dependencies can be a pain in the ass. It's also remarkably bad about telling you how to satisfy the dependencies.

* Error in line 392

After an hour of searching, the solution is:

"Just install these 100 packages. Package 1-20 need to be compiled from source, because the project requires an old version. And they all have dependencies of their own."