News for hackers. Stuff that matters.
Re: by gopisuvanam on Sunday December 29, 2024 @11:08PM (#42546380)
Running Large Language Models (LLMs) directly in the browser is now possible, thanks to WebLLM, which leverages WebGPU and other modern browser technologies. This brings powerful AI capabilities to client-side applications without the need for a backend, ensuring privacy, low latency, and portability. In this article, we’ll explore how to set up and use WebLLM in a browser-based JavaScript application, including examples of how to run a model, handle chat completions, and enable streaming responses.

Use the link to experiment with working code for LLM in the Browser. Note: you will need GPUs on the device.