Re:Web LLM – WebGPU Powered Inference of Large Language Models
by ranguna on Sunday April 16, 2023 @04:35AM
(#35588253)
I don't get where your question is coming from, you can already run LLMs on a single machine. Checkout llama.cpp, tabby, text generation webui, gpt4all, AI Dungeon open source models like clover-edition, and know this we gpu based app.
[ Parent
| on HN ]
Re:Web LLM – WebGPU Powered Inference of Large Language Models
by firatsarlar on Monday April 17, 2023 @05:16AM
(#35598407)
The question comes from a kind of confusion. We know the requirements of LLMs. How can we run the hardware it is currently working on, only the big LLM, with an 11Gb graphics card? I really didn't mind!