Jasssss
Show HN: Catcher – An AI web testing tool where most tests never hit the API
Re:Show HN: Find the best local LLM for your hardware, ranked by bench...
on Saturday May 16, 2026 @07:05AM (#48159046)
Nice! Mistral 7B v0.1 is sliding_window: 4096 in the HuggingFace config.json (though v0.2 sets it to null). Gemma 2 alternates sliding window (4096) and full attention every other layer. Both have the field in the model config so maybe you could pull it from the same API you're already using.
Re:Show HN: Find the best local LLM for your hardware, ranked by bench...
on Friday May 15, 2026 @06:27AM (#48146857)
The plan command is clever. How do you handle the VRAM estimation for models with sliding window attention vs full context? Something like Mistral at 32k context uses way less KV cache than Llama at the same context length, but from the README it looks like the estimation is based on a fixed context size. Does it account for that?