Re:Llamafile lets you distribute and run LLMs with a single ...
by rightbyte on Thursday November 30, 2023 @05:46PM
(#38480309)
Really? How could this model not have had front page articles on HN? Is the self-hosted one as good as the one or their website?
Re:Llamafile lets you distribute and run LLMs with a single ...
by eyegor on Friday December 01, 2023 @09:16PM
(#38495214)
Hmm, I just assumed people knew about it. Submitted https://news.ycombinator.com/item?id=38495176
Self hosted 6.7b model is phenomenal in my experience, I actually prefer it to chatgpt a lot of the time. Similar quality code but fewer disclaimers and #todo in my experience. Gpt4 is still a bit better at coding but not by much and it's much slower for me.
3-4gb vram or cpu (1.3b): https://huggingface.co/TheBloke/deepseek-coder-1.3b-instruct...
Alternative for chat (1.3b): https://huggingface.co/TheBloke/evolvedSeeker_1_3-GGUF
Alternative for chat (3b): https://huggingface.co/TheBloke/open-llama-3b-v2-wizard-evol...
6-8gb vram (6.7b): https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct...