Re:Open Assistant – project meant to give everyone access to...
by visarga on Saturday February 04, 2023 @03:51PM
(#34658128)
The small models are usually tested on classification, question answering and extraction tasks, not on open text generation where I expect the large models still hold the reign.
Another thread on HN (https://news.ycombinator.com/item?id=34653075) discusses a model that is less than 1B parameters and outperforms GPT-3.5. https://arxiv.org/abs/2302.00923
These models will get smaller and more efficiently use the parameters available.