Re:DeepSeek-R1
by kridsdale1 on Tuesday January 21, 2025 @03:51AM
(#42777847)
That would be a true Mixture of Experts.
I sometimes put the 4 biggest models like this to converge on an optimal solution
I sometimes put the 4 biggest models like this to converge on an optimal solution
Maybe we need a dozen LLMs with different biases. Let them try to convince the main reasoning LLM that it’s wrong in various ways.
Or just have an LLM that is trained on some kind of critical thinking dataset where instead of focusing on facts it focuses on identifying assumptions.