News for hackers. Stuff that matters.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by freedmand on Monday March 13, 2023 @03:17PM (#35140980)
The instruction tuning dataset is only 52,000 rows. It shouldn't be too hard to crowdsource high-quality human answers to this many questions and retrain the model, nixing the dependency on OpenAI.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by Tiberium on Monday March 13, 2023 @04:04PM (#35141698)
Such a thing already exists and there were some results - https://open-assistant.io

I'm not sure why the authors of Alpaca didn't try to train it on this dataset.

Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by IanCal on Monday March 13, 2023 @04:39PM (#35142195)
That dataset isn't released yet.

> Can I download the data?

> You will be able to, under CC BY 4.0, but it's not released yet. We want to remove spam and PII before releasing it.

Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by losteric on Monday March 13, 2023 @05:11PM (#35142734)
There's the OIG dataset (https://laion.ai/blog/oig-dataset/) which was used to train a NeoX 20B ChatBot (https://huggingface.co/togethercomputer/GPT-NeoXT-Chat-Base-...). The dataset is larger and publicly available. I want to try finetuning LLaMa on this tonight.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by Jack5500 on Tuesday March 14, 2023 @09:06AM (#35150827)
how did it go?
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by ilaksh on Monday March 13, 2023 @05:19PM (#35142887)
Wow.. I really hope someone will train this model with that dataset. Or maybe open assistant will pick it up. The results looks so promising.