News for hackers. Stuff that matters.

A comment on StableDiffusion can now run directly in the browser on WebGPU. Parent comment

Re:StableDiffusion can now run directly in the browser on We... by FL33TW00D on Tuesday July 18, 2023 @06:57AM (#36770405)
ONNX is bloated! I got some LLMs working on my own Rust + WebGPU framework a few months ago: https://summize.fleetwood.dev/

I've since moved away from ONNX and to a more GGML style.

[ Parent | on HN ]
Re:StableDiffusion can now run directly in the browser on We... by taminka on Tuesday July 18, 2023 @08:16AM (#36771017)
what's the difference between onnx and ggml style?
[ Parent | on HN ]
Re:StableDiffusion can now run directly in the browser on We... by FL33TW00D on Tuesday July 18, 2023 @09:09AM (#36771563)
ONNX consumes a .onnx file, which is a definition of the network and weights. GGML instead just consumes the weights, and defines the network in code.

Being bound to ONNX means moving at a slower velocity - the field moves so fast that you need complete control.

[ Parent | on HN ]
Re:StableDiffusion can now run directly in the browser on We... by michaelmior on Tuesday July 18, 2023 @09:31AM (#36771869)
I haven't used ONNX or GGML, but presumably using GGML means you need to reimplement the network architecture?
[ Parent | on HN ]
Re:StableDiffusion can now run directly in the browser on We... by FL33TW00D on Tuesday July 18, 2023 @11:54AM (#36774485)
You do! But it offers quite a fluid API making it pretty simple. You can see my attempt at a torchesque API here: https://twitter.com/fleetwood___/status/1679889450623459328
[ Parent | on HN ]
Re:StableDiffusion can now run directly in the browser on We... by naillo on Tuesday July 18, 2023 @09:51AM (#36772216)
Do you have any good resources or links on using ggml with wasm?
[ Parent | on HN ]
Re:StableDiffusion can now run directly in the browser on We... by bkitano19 on Tuesday July 18, 2023 @11:16AM (#36773856)
Hey! This is what I've been working on, would love to chat, feel free to email
[ Parent | on HN ]
Re:StableDiffusion can now run directly in the browser on We... by FL33TW00D on Tuesday July 18, 2023 @11:55AM (#36774507)
Sure! My email is in my profile.
[ Parent | on HN ]
Re:StableDiffusion can now run directly in the browser on We... by FL33TW00D on Tuesday July 18, 2023 @11:55AM (#36774501)
I think the Whisper example is your best bet! https://github.com/ggerganov/whisper.cpp/tree/master/example...
[ Parent | on HN ]