News for hackers. Stuff that matters.
Re:Inkling 2 Mini by hedgehog on Friday November 20, 2026 @08:38PM (#50584949)
Seems pretty promising in a few quick coding tests. 27 tokens/s decode on Ryzen 395 and llama.cpp Vulkan. Besides the obvious unmerged PRs (32694, 32657, 32429, 32704) you'll want below tiny patch for about 30% speed boost.

Also, if anyone from the team is reading, if you have access for researchers I'd be interested in doing some higher volume tests & harness tuning than I have hardware for personally.

    --- a/src/llama-graph.cpp
    +++ b/src/llama-graph.cpp
    @@ -2670,10 +2670,12 @@ ggml_tensor \* llm_graph_context::build_moe_ffn(
                         const float beta = hparams.situ_beta;
                         const float lb   = hparams.situ_linear_beta;

    -                    ggml_tensor \* act = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, cur, 1.0f/beta)), beta);
    +                    // gate/up are strided views of the fused gate_up output; ggml-vulkan's SCALE
    +                    // needs a contiguous src, so without the cont both scales fall back to the CPU
    +                    ggml_tensor \* act = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, ggml_cont(ctx0, cur), 1.0f/beta)), beta);
                         act = ggml_mul(ctx0, act, ggml_sigmoid(ctx0, cur));
                         if (lb > 0.0f) {
    -                        up = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, up, 1.0f/lb)), lb);
    +                        up = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, ggml_cont(ctx0, up), 1.0f/lb)), lb);
                         }
                         cur = ggml_mul(ctx0, act, up);
                         cb(cur, "ffn_moe_situ", il);
Re:Inkling 2 Mini by porkloin on Friday November 20, 2026 @09:39PM (#50585363)
Tested on a 7900xtx, I got a similar (~35%) improvement. Thanks for sharing!