News for hackers. Stuff that matters.

A comment on Pro Max 5x quota exhausted in 1.5 hours despite moderate usage. Parent comment

Re:Pro Max 5x quota exhausted in 1.5 hours despite moderate ... by yumraj on Sunday April 12, 2026 @08:09PM (#47745949)
Where are you getting 60GB from? It shouldn’t be that large.

But yes, would love to save context/cache such that it can be played back/referred to if needed.

/compact is a little black box that I just have to trust that is keeping the important bits.

[ Parent | on HN ]
Re:Pro Max 5x quota exhausted in 1.5 hours despite moderate ... by davmre on Sunday April 12, 2026 @10:07PM (#47746723)
The KV cache consists of activation vectors for every attention head at every layer of the model for every token, so it gets quite large. ChatGPT also estimates 60-100GB for full token context of an Opus-sized model:

https://chatgpt.com/share/69dc5030-268c-83e8-92c2-6cef962dc5...

[ Parent | on HN ]
Re:Pro Max 5x quota exhausted in 1.5 hours despite moderate ... by visarga on Monday April 13, 2026 @08:20AM (#47750943)
There are ways to quantize or compress KV cache down.
[ Parent | on HN ]
Re:Pro Max 5x quota exhausted in 1.5 hours despite moderate ... by CraigRood on Monday April 13, 2026 @09:24AM (#47751633)
That is actually nuts.... I'm trying to understand the true costs of AI, wonder how I plug this in!
[ Parent | on HN ]