Qwen3.6-27B (not 3.8) was a much better coder than Gemma4-31B, yet Gemma4-31B was a much better reasoner and general LLM to talk with.

Sure, 3.8 maybe it's better now, but an accurate comparison would be with a new Gemma4-31B iteration (that doesn't exist).

I guess the question is how much of the coding skill gap between qwen3.6-27b and gemma4-31b is down to the model architecture, and how much is because of the training data.

Assuming you can tweak the training data, regenerate qwen3.6, and get a better coder, then presumably you could have variants - e.g. qwen3.6-swift-27b and qwen3.6-python-27b. Or maybe all coding is too intertwined and you can only get splits like qwen3.6-research-27b and qwen3.6-coding-27b. Which isn't quite my pluggable-models dream, but it's a step closer.

But maybe the difference isn't the training data, it's the architecture, in which case pluggable models is probably not possible.