Re:Web LLM – WebGPU Powered Inference of Large Language Models
by brucethemoose2 on Saturday April 15, 2023 @04:21PM
(#35584179)
The GPU's ML task energy is so much lower that you'd probably get better performance running everything on the GPU.
I think some repos have tried splitting things up between the NPU and GPU as well, but they didn't get good performance out of that combination? Not sure why, as the NPU is very low power.
I think some repos have tried splitting things up between the NPU and GPU as well, but they didn't get good performance out of that combination? Not sure why, as the NPU is very low power.