Zerostack – A Unix-inspired coding agent written in pure Rust

Re:Zerostack – A Unix-inspired coding agent written in pure ... by vlovich123 on Saturday May 16, 2026 @11:02PM (#48165714)

It has nothing to do with local RAM usage. But a million tokens of LLM context is decidedly not 5mb.

The rough estimate is 2 * L * H_kv * D * bytes per element

Where:

* L = number of layers * H_kv = # of KV heads * D = head dimension * factor of 2 = keys + values

The dominant factor here is typically 2 * H_kv * D since it’s usually at least 2048 bytes. Per token.

For Llama3 7B youre looking at 128gib if you’re context is really 1M (not that that particular model supports a context so big). DeepSeek4 uses something called sparse attention so the above calculus is improved - 1M of context would use 5-10GiB.

But regardless of the details, you’re off by several orders of magnitude.

[ Parent | on HN ]

Re:Zerostack – A Unix-inspired coding agent written in pure ... by tujux on Saturday May 16, 2026 @11:19PM (#48165778)

Pretty sure we're talking about the output text, not the tensors.

[ Parent | on HN ]

Re:Zerostack – A Unix-inspired coding agent written in pure ... by m00x on Sunday May 17, 2026 @03:38AM (#48166827)

These LLM replies are really getting annoying.

[ Parent | on HN ]

Re:Zerostack – A Unix-inspired coding agent written in pure ... by vlovich123 on Sunday May 17, 2026 @10:48AM (#48169423)

Mine? I literally wrote what I wrote because “context window” as a term of art refers to the LLM’s context window.

I guess get better at detecting LLMs instead of accusing everything of being an LLM reply?

[ Parent | on HN ]