Re:PaliGemma: Open-Source Multimodal Model by Google
by KaoruAoiShiho on Wednesday May 15, 2024 @03:40PM
(#40371421)
Isn't this outdated already. It's 2 models slapped together rather than natively multimodal.
[ Parent
| on HN ]
Re:PaliGemma: Open-Source Multimodal Model by Google
by maciejgryka on Wednesday May 15, 2024 @04:11PM
(#40371780)
Isn't "two models slapped together" basically how all of these things work, starting with CLIP? Not sure about GPT4o, obviously, I don't think they released any underlying architecture details?
Re:PaliGemma: Open-Source Multimodal Model by Google
by whimsicalism on Wednesday May 15, 2024 @04:12PM
(#40371793)
Your understanding is correct, even GPT4o will have an encoder model.
Re:PaliGemma: Open-Source Multimodal Model by Google
by tkellogg on Wednesday May 15, 2024 @04:22PM
(#40371920)
what even is “a model”? I’m not sure there is a technical definition that corresponds to how it’s used by the tech public
- single interconnected neural network (LLM attention layers break this, autoencoders complicate this)
- single training pass (LLMs have multiple passes, GANs have a single but produce multiple models)
Re:PaliGemma: Open-Source Multimodal Model by Google
by whimsicalism on Wednesday May 15, 2024 @05:08PM
(#40372371)
> - single training pass (LLMs have multiple passes, GANs have a single but produce multiple models)
LLMs have multiple passes? wdym?
Re:PaliGemma: Open-Source Multimodal Model by Google
by whimsicalism on Wednesday May 15, 2024 @04:12PM
(#40371788)
No, that's how all of these models work.
Re:PaliGemma: Open-Source Multimodal Model by Google
by JeremyHerrman on Wednesday May 15, 2024 @04:38PM
(#40372092)
From TFA, PaliGemma is competitive to GPT-4o and even beats it in terms of speed and OCR accuracy. It also can do object detection (bounding boxes) and segmentation which GPT-4V/o and Claude 3 Opus can't do at all.
Not to mention it's built to be fine tuned and commercially permissive!