News for hackers. Stuff that matters.

A comment on Ask HN: What hardware are you using to run local LLMs?. Parent comment

Re:Ask HN: What hardware are you using to run local LLMs? by jonaustin on Monday November 02, 2026 @09:34PM (#50347273)
Try out qwen.8 flash next via omlx/splash.

The Vardsan97 balanced ple8 version is quite good and fast.

[ Parent | on HN ]
Re:Ask HN: What hardware are you using to run local LLMs? by 1broseidon on Tuesday November 03, 2026 @12:44AM (#50348526)
I’ll check it out. I tried the unsloth version but it couldn’t handle anything larger than an 8k context window unless I used a 2 or 3 bit quant
[ Parent | on HN ]