News for hackers. Stuff that matters.
Re:RedPajama 7B (an Apache 2.0-licensed LLaMa) is now available by bestcoder69 on Tuesday June 06, 2023 @03:09PM (#36217635)
Definitely don't quantize to 4-bit at this param count, if that's what you're doing. The redpajama.cpp repo is a bit crazy for making that the default.

I'd say at this size don't count on accuracy. Think: fancy auto-complete, simple NLP task-doer, or a base for fine-tuning a task specific model. IME RedPJ-3B was too small to do anything at all other than ramble. The novel contribution of this model here being you can use this at work, and it has a GGML implementation unlike Falcon and MPT (last I checked) - meaning CPU-based inference on the cloud VM you use at work, maybe.

Re:RedPajama 7B (an Apache 2.0-licensed LLaMa) is now available by redox99 on Tuesday June 06, 2023 @03:43PM (#36218102)
People that run 7B 4bit is probably because they don't have the memory for more. And there are tests showing that quantizing a bigger model is always better than running a smaller model full precision (i.e LlaMa 13B 4bit > 7B fp16)
Re:RedPajama 7B (an Apache 2.0-licensed LLaMa) is now available by anon373839 on Tuesday June 06, 2023 @07:05PM (#36220319)
> a base for fine-tuning a task specific model

Yes. The 7B size can perform very well when tuned on a specific menu of tasks:

1. Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks

https://huggingface.co/papers/2305.14201

2. Gorilla: Large Language Model Connected with Massive APIs

https://arxiv.org/abs/2305.15334