Re:RedPajama 7B (an Apache 2.0-licensed LLaMa) is now available
by redox99 on Tuesday June 06, 2023 @03:43PM
(#36218102)
People that run 7B 4bit is probably because they don't have the memory for more. And there are tests showing that quantizing a bigger model is always better than running a smaller model full precision (i.e LlaMa 13B 4bit > 7B fp16)
Re:RedPajama 7B (an Apache 2.0-licensed LLaMa) is now available
by anon373839 on Tuesday June 06, 2023 @07:05PM
(#36220319)
> a base for fine-tuning a task specific model
Yes. The 7B size can perform very well when tuned on a specific menu of tasks:
1. Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks
https://huggingface.co/papers/2305.14201
2. Gorilla: Large Language Model Connected with Massive APIs
I'd say at this size don't count on accuracy. Think: fancy auto-complete, simple NLP task-doer, or a base for fine-tuning a task specific model. IME RedPJ-3B was too small to do anything at all other than ramble. The novel contribution of this model here being you can use this at work, and it has a GGML implementation unlike Falcon and MPT (last I checked) - meaning CPU-based inference on the cloud VM you use at work, maybe.