compinter
Re:QuIP#: 2-bit Quantization for LLMs
on Saturday December 09, 2023 @06:20AM (#38580936)
So, just like the rest of my code then
Re:MLX: An array framework for Apple Silicon
on Wednesday December 06, 2023 @03:40PM (#38549205)
We aren't quite there yet, but the last year has been an incredibly exciting time for the open source LLM community. If your computer is decently powerful, you might be really surprised by what's already possible. LM Studio on Apple Silicon supports GGUF quants on GPU out the box.
Re:LM Studio – Discover, download, and run local LLMs
on Thursday November 23, 2023 @03:05AM (#38390548)
For me it's pretty simple- LM Studio supports Apple Silicon GPU acceleration out of the box, and I like the interface better than Gradio Web UI. It saves me the headache and the tinkering of the alternatives. That said, free software that's hiring developers probably won't stay free for long, so I'm keeping my eye on other options.
Tl;DR It's for Mac users