Re:Web LLM – WebGPU Powered Inference of Large Language Models
by paulgb on Saturday April 15, 2023 @08:32PM
(#35586065)
I wouldn’t consider this to be “misusing” a graphics API in the same sense that tensor operations on top of WebGL are. WebGPU intentionally provides compute shaders with the goal of enabling non-graphical pipelines.
> [WebGPU use cases include] Executing machine learning models efficiently on the GPU. It is possible to do general-purpose GPU (GPGPU) computation in WebGL, but it is sub-optimal and much more difficult.
Re:Web LLM – WebGPU Powered Inference of Large Language Models
by rektide on Saturday April 15, 2023 @10:13PM
(#35586638)
I'm more worried that it'll end up kind of inferior to like, Google's other work, OpenXLA & iree, which seem to be the cross-target ML toolkit.
https://news.ycombinator.com/item?id=35078410
I really want this to be good. It's hard to really trust the web spec is going to live up to what the other really really good web ML work is. But it'll at least unlock some good speed up, whenever it lands. I genuinely don't mind more delay, if it helps get things into a better position for long term wins.
Re:Web LLM – WebGPU Powered Inference of Large Language Models
by archerx on Sunday April 16, 2023 @03:28AM
(#35588010)
This is really cool and I had no idea about it. I tried the live style transfer on live video demo and was amazed, even the with gpu polyfill I was getting 5fps on my underpowered laptop. It brought me back to the days of deepdream and deepstyle but live and on the web.
https://www.w3.org/TR/webnn/
Seems to be in development over at Chrome: https://chromestatus.com/feature/5738583487938560