Re:Google Gemini: Context Caching
by nancarrow on Thursday May 16, 2024 @04:05PM
(#40382535)
"some" is doing a lot of lifting. # of tokens * # of layers * head dimension * # of heads * 2 (K+V vectors) * 4-16bits (depending on quantization)
[ Parent
| on HN ]
There are no comments on this comment yet.