pseudollm
Re:Gemma 4 12B: A unified, encoder-free multimodal model
on Wednesday June 03, 2026 @08:01PM (#48391855)
No there isn't - read the paper. It's just 40msec raw audio samples. Multiplied by one matrix to translate to 3800 input vector. That's it. The next 40 msec are fed in the next transformer input step. Without any positional encoding. Repeat ad infinitum
Re:Gemma 4 12B: A unified, encoder-free multimodal model
on Wednesday June 03, 2026 @07:52PM (#48391775)
> usefulness of the RTX Spark
Not really. There's a reason the announcement didn't include ANY benchmark (!) and didn't mention EXACTLY what is the memory bandwidth. It's going to be dog-slow unusable for large models, as tok/sec is basically bandwidth divided by active weights. Rumoured 300GB/s / 30GB active weights (decent model) = 10 tokens per second, which is really slow