You can already run ML models with GPU acceleration in WebGL using tensorflow.js and other libraries. WebGPU will make things better for sure, but I think the major obstacles to running large models like Stable Diffusion will not be solved by WebGPU.

WebGPU will not expose Nvidia-specific tensor cores, at least initially. But the main issues are with loading and processing gigabytes of data in browsers, which aren't addressed by WebGPU at all. You'll have difficulty downloading and storing all that data and quickly run into out of memory crashes.

In principle, yes. With webGPU (but also WebGL2) we get access to compute shaders, which makes it feasible/easier to evaluate a model like Stable Diffusion.

The biggest issue I see is that those models (or rather their trained parameters) are usually pretty large (several GiB). So it'll take a while to set up in the browser before the evaluation can actually start. It'll also require a lot of bandwidth on both ends.

A lot of those things should already be doable with the fragment shaders we get from WebGL and a lot of hackery, like clever (ab)use of textures. So the actual issue that we're not seeing a lot of this is probably not due to it being impossible right now...

> The biggest issue I see is that those models (or rather their trained parameters) are usually pretty large (several GiB). So it'll take a while to set up in the browser before the evaluation can actually start. It'll also require a lot of bandwidth on both ends.

Might not be feasible due to memory constraints (I'm not sure), but browsers can load data from disk into memory without having to touch the network. So you could in theory ask the users to download the model separately, then ask them to select it from a file picker, and the browser can have access to it that way.

Next step: browser JS VMs need to allow you access gigs of RAM in-memory, it's surprisingly locked down

It'd be nice if the models could be loaded via WebTorrent[1], which would significantly reduce the bandwidth requirement for whomever releases it.

[1] https://github.com/webtorrent/webtorrent

The models are certainly available on torrent though

Right. Another option is running the model locally from a web page, served from the project source directory, which is still a lot easier than setting up a platform specific GPU accelerated ML dev environment.

WebGL2 doesn't support compute shaders.

It did for a short while as an extension, then Google decided to throw away Intel's work and only support compute in WebGPU.

It's a replacement for WebGL, but the key difference is that it supports compute shaders.

It also supports many more features in rendering which previously were unavailable or so badly supported you couldn't use them. It's basically like going from 2005 to 2015 in terms of capabilities.

There are still notable gaps, like no bindless resources or raytracing though. So don't expect e.g. unreal engine to run on WebGPU without some significant compromises.

Someone patched Chrome to enable raytracing with WebGPU: https://github.com/maierfelix/chromium-ray-tracing

WebGL2 also supports them! And now that Apple finally added support (like half a year ago(?)) to it, it can be fairly safely used

I believe WebGL compute shader development has been halted[1], in favor of making WebGPU happen. Though it's possible to run it in dev builds in some cases, I'm pretty sure there's no browser that has it on by default, and likely this won't happen.

Apple does support WebGL2, but compute shader support is not part of that core spec. The demos[2] certainly don't work in Safari in my quick test.

[1]: https://registry.khronos.org/webgl/specs/latest/2.0-compute/

[2]: https://9ballsyndrome.github.io/WebGL_Compute_shader/webgl-c...

Thanks Google that dropped them out of Chrome and we all know how the Web is basically whatever Chrome does, so WebGPU it is.

Is it a replacement? It looks more like a framework that lives on top of webgl.

You shouldn't be downvoted, your question is reasonable even if it demonstrates a misunderstanding. So it's not a wrapper over WebGL, it's a more modern, lower-level GPU programming model. There are equivalent efforts in native environments to replace OpenGL - notably Vulkan and (Apple's) Metal.

As far as I understand, the approach allows far more flexibility, at the expense of higher complexity. It's less "stateful" than WebGL, which basically gives you a big class that manages everything OOP-style.

It's the other way around. WebGPU is to Vulkan what WebGL is to OpenGL. It's less low level than vulkan though.

WebGPU is to WebGL as Vulkan is to OpenGL, a lower level, low overhead graphics API.