News for hackers. Stuff that matters.

A comment on Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook. Parent comment

Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by satvikpendem on Thursday March 16, 2023 @08:41PM (#35191131)
I'm trying the 13B one, it's decent, I'd say on par with GPT 3, definitely not 3.5 or 4 yet however. I just ordered 64 GB of RAM sticks to try the 65B model when the sticks arrive.
[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by dvt on Thursday March 16, 2023 @08:51PM (#35191230)
How well does it work on DRAM? I thought everyone running heavier models are doing it on GPUs.
[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by satvikpendem on Thursday March 16, 2023 @09:12PM (#35191413)
Alpaca works entirely off CPU so the models should work fine. I'll try tomorrow and report back.
[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by BoorishBears on Friday March 17, 2023 @04:28AM (#35194581)
Recent advances in ML have finally redeemed my decision to building a gaming PC with 64GB of RAM and a 3090 when my most played game was Dolphin Emulator.
[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by bdsa on Friday March 17, 2023 @05:16AM (#35194859)
I forgot briefly that Dolphin refers to GameCube/Wii and was very curious about what a "dolphin emulator" would be (and how it would differ from a dolphin simulator)
[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by lbltavares on Saturday March 18, 2023 @08:07PM (#35214527)
LOL same here
[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by Magicianx69 on Monday March 20, 2023 @02:12AM (#35227887)
man i only play Minecraft an i didn't bother setting it up to use gpu it friggin runs in cpu. so my gpu is not even woken up from slumber yet
[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by unshavedyak on Friday March 17, 2023 @10:10AM (#35197294)
How are you trying it? Alpaca? Or just straight llama.cpp?
[ Parent | on HN ]
Re:Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style... by satvikpendem on Friday March 17, 2023 @12:04PM (#35198890)
There's a project called Dalai with a web frontend so you can call npx dalai llama (heh) to get it running. But of course they're all running LLaMA underneath.
[ Parent | on HN ]