News for hackers. Stuff that matters.
Re:Web LLM – WebGPU Powered Inference of Large Language Models by bhouston on Saturday April 15, 2023 @05:22PM (#35584654)
It is funny how we have WebNN API coming, but it is so slow to arrive that we are misusing graphics APIs to do NN again. Anyone have a clue when WebNN API will arrive? It would be significantly more power efficient than using WebGPU/WebGL.

https://www.w3.org/TR/webnn/

Seems to be in development over at Chrome: https://chromestatus.com/feature/5738583487938560

Re:Web LLM – WebGPU Powered Inference of Large Language Models by paulgb on Saturday April 15, 2023 @08:32PM (#35586065)
I wouldn’t consider this to be “misusing” a graphics API in the same sense that tensor operations on top of WebGL are. WebGPU intentionally provides compute shaders with the goal of enabling non-graphical pipelines.

> [WebGPU use cases include] Executing machine learning models efficiently on the GPU. It is possible to do general-purpose GPU (GPGPU) computation in WebGL, but it is sub-optimal and much more difficult.

https://gpuweb.github.io/gpuweb/explainer/

Re:Web LLM – WebGPU Powered Inference of Large Language Models by rektide on Saturday April 15, 2023 @10:13PM (#35586638)
I'm more worried that it'll end up kind of inferior to like, Google's other work, OpenXLA & iree, which seem to be the cross-target ML toolkit. https://news.ycombinator.com/item?id=35078410

I really want this to be good. It's hard to really trust the web spec is going to live up to what the other really really good web ML work is. But it'll at least unlock some good speed up, whenever it lands. I genuinely don't mind more delay, if it helps get things into a better position for long term wins.

Re:Web LLM – WebGPU Powered Inference of Large Language Models by archerx on Sunday April 16, 2023 @03:28AM (#35588010)
This is really cool and I had no idea about it. I tried the live style transfer on live video demo and was amazed, even the with gpu polyfill I was getting 5fps on my underpowered laptop. It brought me back to the days of deepdream and deepstyle but live and on the web.