News for hackers. Stuff that matters.
Re:Locally hosted LLM use cases? by brucethemoose2 on Monday June 26, 2023 @11:47PM (#36488662)
13B finetunes are quite "smart" at what they do and can fit on modern phones. 7B is reasonably fast running Vulkan/Apache TVM via mlc-llm, not sure if 13B has a profile yet.

Check out the 13B finetunes on HF: https://huggingface.co/models?sort=modified&search=13b

There are no comments on this comment yet.