News for hackers. Stuff that matters.

A comment on Jamesob's guide to running SOTA LLMs locally. Parent comment

Re:Jamesob's guide to running SOTA LLMs locally by hypfer on Friday July 03, 2026 @01:57PM (#48777876)
That math (250k context, Q4 model, 24GB VRAM) only checks out at q4 quant for the K/V cache, which is probably not the best idea.
[ Parent | on HN ]

There are no comments on this comment yet.