News for hackers. Stuff that matters.
Re:PaliGemma: Open-Source Multimodal Model by Google by whimsicalism on Wednesday May 15, 2024 @09:36PM (#40374380)
llava-mistral v1.6 is apache, my friend :)
Re:PaliGemma: Open-Source Multimodal Model by Google by codester1000 on Wednesday May 15, 2024 @10:59PM (#40374791)
Where did you get this info? because I would love to use it, but I went through their info and it says they use a lot of the same data as 1.5 and the acknowledgement section of their site says: "The dataset is CC BY NC 4.0 (allowing only non-commercial use) and models trained using the dataset should not be used outside of research purposes"

Would love it if I could use LLaVA, but don't want to spend the money on like 18 A100s for 24 hrs that they use for training it. A lot of the models using CC BY NC 4.0 datasets, like VILA, thats not available for commercial use unless you train the model yourself. This is the first time at least a research or company has been open with this info, they specifically say: only the pt models can be used with fine-tuning for commercial use.

Re:PaliGemma: Open-Source Multimodal Model by Google by llama_person on Thursday May 16, 2024 @01:37AM (#40375518)
you can try out https://huggingface.co/datasets/BAAI/SVIT which appears to support commercial. I've not tried it yet, but it seems to be an option.

If you build a smaller model, you should need much less than 18 A100s for 24 hours, though I don't disagree you'll need at least a few.

Re:PaliGemma: Open-Source Multimodal Model by Google by whimsicalism on Thursday May 16, 2024 @12:36PM (#40380334)
The author has added the weights on huggingface under Apache-2.0 license [0]. All previous versions to 1.6 were not listed under Apache. This repo has no code, it is the weights repository.

[0]: https://huggingface.co/liuhaotian/llava-v1.6-mistral-7b

Re:PaliGemma: Open-Source Multimodal Model by Google by codester1000 on Sunday May 19, 2024 @02:44AM (#40404755)
Oh awesome, thanks so much, didn't see this
Re:PaliGemma: Open-Source Multimodal Model by Google by codester1000 on Wednesday May 15, 2024 @10:37PM (#40374681)
yes their code is, but not their dataset if you want to use the model weights without training yourself