Gemma 4 12B: A unified, encoder-free multimodal model

Re:Gemma 4 12B: A unified, encoder-free multimodal model by make3 on Wednesday June 03, 2026 @01:33PM (#48387006)

I guarantee you there's positional information one way or another. they just don't mention it because positional embeddings are extremely cheap computationally, not worth mentioning

[ Parent | on HN ]

Re:Gemma 4 12B: A unified, encoder-free multimodal model by neosat on Wednesday June 03, 2026 @01:39PM (#48387096)

Agree. Audio has strongly temporal so there is almost certainly some positional encoding one way or another.

[ Parent | on HN ]

Re:Gemma 4 12B: A unified, encoder-free multimodal model by mchinen on Wednesday June 03, 2026 @01:52PM (#48387294)

Ah yeah, thinking further it's probably just using some positioning embedding based on sequence numbering added in the LLM layers. For vision it needs the patch location as well.

[ Parent | on HN ]

Re:Gemma 4 12B: A unified, encoder-free multimodal model by aesthesia on Wednesday June 03, 2026 @06:07PM (#48390796)

Audio is 1 dimensional so the usual RoPE position encoding should handle it like it does for text tokens. You only need extra position encoding for higher-dimensional stuff like images.

[ Parent | on HN ]

Re:Gemma 4 12B: A unified, encoder-free multimodal model by pseudollm on Wednesday June 03, 2026 @08:01PM (#48391855)

No there isn't - read the paper. It's just 40msec raw audio samples. Multiplied by one matrix to translate to 3800 input vector. That's it. The next 40 msec are fed in the next transformer input step. Without any positional encoding. Repeat ad infinitum

[ Parent | on HN ]