The Coming of Local LLMs

Re:The Coming of Local LLMs by kbrkbr on Tuesday April 11, 2023 @01:18PM (#35528145)

In my opinion the problem with these is engineering a good prompt. I read of lots of people only getting nonsense or repetitions, and learned a bit from what they shared. These models are not chat bots.

Vicuna is more friendly in that regard.

But I’m well aware of their limitations also, and I can see how one can be underwhelmed. They are not jacks of all trades

[ Parent | on HN ]

Re:The Coming of Local LLMs by whimsicalism on Tuesday April 11, 2023 @01:21PM (#35528181)

Alpaca & Alpaca-LoRA are literally trained to be chat bots. Their goal is to be instruct-tuned.

[ Parent | on HN ]

Re:The Coming of Local LLMs by MacsHeadroom on Wednesday April 12, 2023 @12:37AM (#35535058)

Alpaca-LoRA, and all LoRas, are garbage. Alpaca is horrible compared to newer finetunes. Even cleaning the Alpaca dataset and retraining a cleaned Alpaca improves its performance greatly.

But newer finetunes like Vicuna go well beyond that, including hundreds of thousands of real human conversations with GPT-4 ChatGPT in the dataset (unlike Alpaca's fully synthetic dataset).

Vicuna-13B in 16bit is easily comparable to ChatGPT-3.5 in capability. Newer finetunes coming out nearly every day are going beyond chatGPT-3.5 and getting closer and closer to GPT-4 performance.

You don't even have to install anything to validate this for yourself. There's a live web demo of Vicuna-13B right here: https://chat.lmsys.org/ (disable ad blocker if it does not load)

[ Parent | on HN ]

Re:The Coming of Local LLMs by MacsHeadroom on Wednesday April 12, 2023 @03:16AM (#35536050)

Also, InstructGPT was released in January 2021 and Vicuna-13B blows it out of the water.

[ Parent | on HN ]