News for hackers. Stuff that matters.
Re:Google Gemini: Context Caching by nancarrow on Thursday May 16, 2024 @04:05PM (#40382535)
"some" is doing a lot of lifting. # of tokens * # of layers * head dimension * # of heads * 2 (K+V vectors) * 4-16bits (depending on quantization)

There are no comments on this comment yet.