News for hackers. Stuff that matters.
Re:PaliGemma: Open-Source Multimodal Model by Google by simonw on Wednesday May 15, 2024 @05:12PM (#40372414)
Have you seen any good documentation anywhere on how to do that?
Re:PaliGemma: Open-Source Multimodal Model by Google by llama_person on Wednesday May 15, 2024 @05:33PM (#40372667)
Here's a tutorial https://wandb.ai/byyoung3/ml-news/reports/How-to-Fine-Tune-L...

There's not really a super easy to use software solution yet, but a few different ones have cropped up. Right now you'll have to read papers to get the training recipes.

- https://github.com/haotian-liu/LLaVA/blob/main/scripts/finet...

- https://github.com/InternLM/xtuner/tree/main

- https://github.com/TinyLLaVA/TinyLLaVA_Factory

Is a pointer in the right direction, along with:

https://arxiv.org/abs/2304.08485

Re:PaliGemma: Open-Source Multimodal Model by Google by simonw on Wednesday May 15, 2024 @05:45PM (#40372786)
Thanks!
Re:PaliGemma: Open-Source Multimodal Model by Google by whimsicalism on Wednesday May 15, 2024 @07:00PM (#40373398)
the linked resources are good, also the llava repo has pretty good guides on how to train which you can adapt to SFT