News for hackers. Stuff that matters.

A comment on Qwen3-Coder-Next. Parent comment

Re:Qwen3-Coder-Next by cgearhart on Tuesday February 03, 2026 @04:15PM (#46877397)
Any notes on the problems with MLX caching? I’ve experimented with local models on my MacBook and there’s usually a good speedup from MLX, but I wasn’t aware there’s an issue with prompt caching. Is it from MLX itself or LMstudio/mlx-lm/etc?
[ Parent | on HN ]
Re:Qwen3-Coder-Next by dust42 on Tuesday February 03, 2026 @06:00PM (#46878639)
It is the buffer implementation. [u1 10kTok]->[a1]->[u2]->[a2]. If you branch between the assistant1 and user2 answers then MLX does reprocess the u1 prompt of let's say 10k tokens while llama.cpp does not.

I just tested with GGUF and MLX of Qwen3-Coder-Next with llama.cpp and now with LMStudio. As I do branching very often, it is highly annoying for me to the point of being unusable. Q3-30B is much more usable then on Mac - but by far not as powerful.

[ Parent | on HN ]
Re:Qwen3-Coder-Next by anon373839 on Tuesday February 03, 2026 @06:56PM (#46879257)
There’s this issue/outstanding PR: https://github.com/lmstudio-ai/mlx-engine/pull/188#issuecomm...
[ Parent | on HN ]