Founding AI Engineer at OpenPipe here, using a fine tuned "router LLM" to route between various specialized (inc fine tuned but not necessarily) applied models depending on the input is becoming a common pattern in more modern "graph like" LLM applications.

See LangGraph's "conditional edges" concept here: https://langchain-ai.github.io/langgraph/concepts/low_level/...

You can see how that "routing function" could include a call to a "Router LLM." And yes, fine tuning is a great method to better improve the routing intelligence of said Router LLM.

Great question btw!

Very loosely, isn’t this what is happening inside most LLMs that have a “multi-head” mechanism?

Already a big thing. See the constellation architecture used here:

https://arxiv.org/html/2403.13313v1

Check out https://unify.ai/chat if you're interested in a router optimised for cost/ttft/performance for commercial language models.

Worth mentioning that you don’t even need separate models to implement this. Dynamically loading LoRA adapters is much more efficient, and is the approach Apple took.