Re:Our Smallest, Most Efficient Model: Muse Ember
by GuB-42 on Monday November 09, 2026 @06:26PM
(#50433841)
Human lawyers, coders, accountants and designers are a lot more than their jobs.
They are people with common knowledge, a lawyer has to understand carpentry to understand how to deal with how a client's roof fell off, a coder has to understand transportation for writing a travel app, an accountant has to understand medicine to to accounting for a pharmacy, a designer how to understand cooking to design a toaster. Not expert knowledge, but enough to understand that toasters are hot and need electricity.
A LLM for people working these jobs also need to have common knowledge, that's the whole point of using LLMs, you can prompt them with intrinsically ambiguous and incomplete natural language and they are supposed to fill in the gaps. And the more common knowledge they have, the better they are at it. So why settle for less? If it is an optimization problem, we already have the solution, it is the "mixture of experts", which is better at optimizing then having an "lawyer LLM", "coder LLM", etc... because it is better adapted to the internal structure of the neural network behind it.
They are people with common knowledge, a lawyer has to understand carpentry to understand how to deal with how a client's roof fell off, a coder has to understand transportation for writing a travel app, an accountant has to understand medicine to to accounting for a pharmacy, a designer how to understand cooking to design a toaster. Not expert knowledge, but enough to understand that toasters are hot and need electricity.
A LLM for people working these jobs also need to have common knowledge, that's the whole point of using LLMs, you can prompt them with intrinsically ambiguous and incomplete natural language and they are supposed to fill in the gaps. And the more common knowledge they have, the better they are at it. So why settle for less? If it is an optimization problem, we already have the solution, it is the "mixture of experts", which is better at optimizing then having an "lawyer LLM", "coder LLM", etc... because it is better adapted to the internal structure of the neural network behind it.