Re:PaliGemma: Open-Source Multimodal Model by Google
by simonw on Wednesday May 15, 2024 @05:12PM
(#40372414)
Have you seen any good documentation anywhere on how to do that?
[ Parent
| on HN ]
Re:PaliGemma: Open-Source Multimodal Model by Google
by llama_person on Wednesday May 15, 2024 @05:33PM
(#40372667)
Here's a tutorial https://wandb.ai/byyoung3/ml-news/reports/How-to-Fine-Tune-L...
There's not really a super easy to use software solution yet, but a few different ones have cropped up. Right now you'll have to read papers to get the training recipes.
- https://github.com/haotian-liu/LLaVA/blob/main/scripts/finet...
- https://github.com/InternLM/xtuner/tree/main
- https://github.com/TinyLLaVA/TinyLLaVA_Factory
Is a pointer in the right direction, along with:
Re:PaliGemma: Open-Source Multimodal Model by Google
by simonw on Wednesday May 15, 2024 @05:45PM
(#40372786)
Thanks!
Re:PaliGemma: Open-Source Multimodal Model by Google
by whimsicalism on Wednesday May 15, 2024 @07:00PM
(#40373398)
the linked resources are good, also the llava repo has pretty good guides on how to train which you can adapt to SFT