News for hackers. Stuff that matters.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by redox99 on Monday March 13, 2023 @02:39PM (#35140370)
They used OpenAI davinci to generate the dataset for instruction tuning. They mention one of the reasons this model is noncommercial is because OpenAI's terms of use prohibit developing models that compete with OpenAI.

They also published the generated dataset.

If one were to use this dataset to tune your own commercial model, would this be OK, or illegal? I would guess OpenAIs ToS no longer applies, as this is now Alpaca's dataset. You are never agreeing to OpenAIs ToS.

Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by wongarsu on Monday March 13, 2023 @02:43PM (#35140441)
I think it's a reasonable interpretation that Alpaca developers hold copyright over the dataset and can distribute it however they want, just how you hold copyright for any pictures you make with a camera. OpenAI can go after the Alpaca developers for the ToS violation, maybe force them to stop distributing the dataset on those grounds, but I don't see how that would affect third parties using that dataset.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by freedmand on Monday March 13, 2023 @03:17PM (#35140980)
The instruction tuning dataset is only 52,000 rows. It shouldn't be too hard to crowdsource high-quality human answers to this many questions and retrain the model, nixing the dependency on OpenAI.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by Tiberium on Monday March 13, 2023 @04:04PM (#35141698)
Such a thing already exists and there were some results - https://open-assistant.io

I'm not sure why the authors of Alpaca didn't try to train it on this dataset.

Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by IanCal on Monday March 13, 2023 @04:39PM (#35142195)
That dataset isn't released yet.

> Can I download the data?

> You will be able to, under CC BY 4.0, but it's not released yet. We want to remove spam and PII before releasing it.

Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by losteric on Monday March 13, 2023 @05:11PM (#35142734)
There's the OIG dataset (https://laion.ai/blog/oig-dataset/) which was used to train a NeoX 20B ChatBot (https://huggingface.co/togethercomputer/GPT-NeoXT-Chat-Base-...). The dataset is larger and publicly available. I want to try finetuning LLaMa on this tonight.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by Jack5500 on Tuesday March 14, 2023 @09:06AM (#35150827)
how did it go?
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by ilaksh on Monday March 13, 2023 @05:19PM (#35142887)
Wow.. I really hope someone will train this model with that dataset. Or maybe open assistant will pick it up. The results looks so promising.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by scottlawson on Monday March 13, 2023 @05:53PM (#35143451)
This model is for non commercial use only so it's unclear to me how it could compete commercially with OpenAI. Sure, you could decide to interpret "compete" in a way that makes it seem like this violates TOS, but the mom commercial restriction is a powerful counterargument too.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by vishal0123 on Monday March 13, 2023 @09:13PM (#35145949)
They allow others to use it for noncommercial use. Other research groups won't have to use openAI APIs for some of the usecases, hence the model is competing.
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by hgsgm on Monday March 13, 2023 @08:10PM (#35145258)
> OpenAI's terms of use prohibit developing models that compete with OpenAI.

People shouldn't collaborate with Ope(ratio)nAI's attempt to poison the English language with its name.

Government shouldn't allow lies to be trademarked brand names. It's fraud.