News for hackers. Stuff that matters.
Re:Adept Open Sources 8B Multimodal Modal Fuyu by Q6T46nT668w6i3m on Wednesday October 18, 2023 @04:46PM (#37934545)
Neat idea! Are the batches encoded as tokens into the input sequence? This is something I really like about the multi-modal PALM papers since it enables the multi-modal tokens to be referenced.
Re:Adept Open Sources 8B Multimodal Modal Fuyu by ekelsen on Wednesday October 18, 2023 @06:36PM (#37935722)
Image patches are projected directly into an embedding that goes into the decoder Transformer. The same thing could be done for audio.