News for hackers. Stuff that matters.

A comment on Using LLaMA with M1 Mac and Python 3.11. Parent comment

Re:Using LLaMA with M1 Mac and Python 3.11 by superkuh on Sunday March 12, 2023 @01:24PM (#35123059)
openai is expensive (ie, ~$25/mo for a gpt3 davinci IRC bot in a realtively small channel that only gets used heavily a few hours a day) and censored. And I'm not just talking won't respond to controversial things. Even innocuous topics are blocked. If you try to use gpt3.5-turbo for 10x less cost it is so bad with censoring itself that it can't even pass a turing test. Plus there's the whole data collection and privacy issue.

I just wish these weren't all articles about how to run it on proprietary mac setups. I'm still waiting for the guides on how to run it on a real PC.

[ Parent | on HN ]
Re:Using LLaMA with M1 Mac and Python 3.11 by behnamoh on Sunday March 12, 2023 @01:28PM (#35123132)
> I'm still waiting for the guides on how to run it on a real PC.

Mac __is__ a real PC.

[ Parent | on HN ]
Re:Using LLaMA with M1 Mac and Python 3.11 by smoldesu on Sunday March 12, 2023 @01:35PM (#35123255)
Until Apple decides otherwise?
[ Parent | on HN ]
Re:Using LLaMA with M1 Mac and Python 3.11 by lolinder on Sunday March 12, 2023 @01:59PM (#35123668)
The actual repo's instructions work perfectly without modification under Linux and WSL2:

https://github.com/ggerganov/llama.cpp

[ Parent | on HN ]
Re:Using LLaMA with M1 Mac and Python 3.11 by superkuh on Sunday March 12, 2023 @08:11PM (#35128627)
Thanks! I managed to get this running on CPU/system RAM on Debian 11 pretty easily.
[ Parent | on HN ]
Re:Using LLaMA with M1 Mac and Python 3.11 by zirgs on Sunday March 12, 2023 @02:15PM (#35123934)
https://github.com/oobabooga/text-generation-webui/wiki/LLaM... Here you go. You'll need a Nvidia GPU with at least 8 GB VRAM though.
[ Parent | on HN ]
Re:Using LLaMA with M1 Mac and Python 3.11 by mark_l_watson on Sunday March 12, 2023 @03:40PM (#35125162)
Thanks! I have a Linux laptop with 16G ram and a 10G NVidai 1070, so I might be good to go.
[ Parent | on HN ]
Re:Using LLaMA with M1 Mac and Python 3.11 by goldenCeasar on Sunday March 12, 2023 @09:00PM (#35129118)
Any examples on running on multiple GPUs?
[ Parent | on HN ]
Re:Using LLaMA with M1 Mac and Python 3.11 by miloignis on Sunday March 12, 2023 @03:54PM (#35125351)
The exact steps that work on a Mac should work on x64 Linux, since the addition of AVX2 support! (Source - I did it last night)
[ Parent | on HN ]