Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by saurik on Thursday March 16, 2023 @06:54PM (#35190082)

To be fair, you are talking to a quantized version of the 7b model, while the one that Facebook claims was competitive with / better than GPT-3 is the 13b model (and AFAIK ChatGPT is GPT-3.5, so maybe you should want to focus on 30b+).

[ Parent | on HN ]

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by satvikpendem on Thursday March 16, 2023 @08:41PM (#35191131)

I'm trying the 13B one, it's decent, I'd say on par with GPT 3, definitely not 3.5 or 4 yet however. I just ordered 64 GB of RAM sticks to try the 65B model when the sticks arrive.

[ Parent | on HN ]

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by dvt on Thursday March 16, 2023 @08:51PM (#35191230)

How well does it work on DRAM? I thought everyone running heavier models are doing it on GPUs.

[ Parent | on HN ]

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by satvikpendem on Thursday March 16, 2023 @09:12PM (#35191413)

Alpaca works entirely off CPU so the models should work fine. I'll try tomorrow and report back.

[ Parent | on HN ]

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by BoorishBears on Friday March 17, 2023 @04:28AM (#35194581)

Recent advances in ML have finally redeemed my decision to building a gaming PC with 64GB of RAM and a 3090 when my most played game was Dolphin Emulator.

[ Parent | on HN ]

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by bdsa on Friday March 17, 2023 @05:16AM (#35194859)

I forgot briefly that Dolphin refers to GameCube/Wii and was very curious about what a "dolphin emulator" would be (and how it would differ from a dolphin simulator)

[ Parent | on HN ]

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by lbltavares on Saturday March 18, 2023 @08:07PM (#35214527)

LOL same here

[ Parent | on HN ]

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by Magicianx69 on Monday March 20, 2023 @02:12AM (#35227887)

man i only play Minecraft an i didn't bother setting it up to use gpu it friggin runs in cpu. so my gpu is not even woken up from slumber yet

[ Parent | on HN ]

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by unshavedyak on Friday March 17, 2023 @10:10AM (#35197294)

How are you trying it? Alpaca? Or just straight llama.cpp?

[ Parent | on HN ]

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by satvikpendem on Friday March 17, 2023 @12:04PM (#35198890)

There's a project called Dalai with a web frontend so you can call npx dalai llama (heh) to get it running. But of course they're all running LLaMA underneath.

[ Parent | on HN ]