Re:WebGPU enables local LLM in the browser. Demo site with A...
by andreinwald on Saturday August 02, 2025 @12:57PM
(#44769198)
It works on small Llama-3.2-1B model, specially for less powerfull GPU devices
[ Parent
| on HN ]
Re:WebGPU enables local LLM in the browser. Demo site with A...
by wongarsu on Saturday August 02, 2025 @02:35PM
(#44770117)
The answer is still terrible for the model size. Maybe it's the 4 bit quantization, smaller models tend to react worse to that
For reference, [1] is what stock quen3-0.6B would answer. Not a perfect answer, but much better at nearly half the number of parameters
1: https://markdownpastebin.com/?id=7ad4ad9f325d4354a858480abdc...
Re:WebGPU enables local LLM in the browser. Demo site with A...
by KTibow on Saturday August 02, 2025 @03:53PM
(#44770815)
It's likely the quantization on "Llama-3.2-1B-Instruct-q4f16_1-MLC". inference.net generated this more coherent answer: https://hst.sh/ovilewofox.md