News for hackers. Stuff that matters.
Re:Web LLM – WebGPU Powered Inference of Large Language Models by jimmySixDOF on Saturday April 15, 2023 @11:33PM (#35587038)
I'm also not sure what the real use case is here other than an intellectual experiment. I may be missing the point but as I understand the world, running a Local LLM has nothing to do with the web in any way while it is doing so and so what's the benefit of WebGPU ? If you are streaming some constantly changing live data 3D experience in real time served over the internet by a website you just clicked on then WebGPU is making life easier. If you have a local pile of training data and weights sitting there on your hard drive for serious crunching then its like tying your shoelaces together just because developers want to have write once run everywhere dx which is nice for them but value sould always favor the user. Happy for someone to change my mind.
Re:Web LLM – WebGPU Powered Inference of Large Language Models by bigdict on Sunday April 16, 2023 @01:20AM (#35587478)
WebGPU is much more portable than CUDA.