News for hackers. Stuff that matters.

A comment on Gemini 3.5 Flash. Parent comment

Re:Gemini 3.5 Flash by smnscu on Wednesday May 20, 2026 @01:59AM (#48203653)
Nice post! You piqued my curiosity, so after a bit of research it turns out that, with techniques like MTP/MLA/CSA, it's quite probable that these models are much more efficient (and maybe bigger? tho 400B sounds about right) than a simple RAM breakdown would suggest.

MTP - https://blog.google/innovation-and-ai/technology/developers-...

MLA - https://machinelearningmastery.com/a-gentle-introduction-to-...

CSA - https://deepseek.ai/blog/deepseek-v4-compressed-attention

[ Parent | on HN ]
Re:Gemini 3.5 Flash by Doxon on Thursday May 21, 2026 @02:14PM (#48226845)
These techniques are used by DeepSeek, and work well with the commodity (NVIDIA) GPU's they use. Google designs their entire AI stack from the custom silicon up. So they have different optimization approaches. (Though Gemma does use MTP)
[ Parent | on HN ]