Re:DeepSeek-Prover-V2
by koakuma-chan on Wednesday April 30, 2025 @03:42PM
(#43849773)
> have seen anecdotes where finetuning a small model on domain data makes the model lose coherence on other topics
This is expected behaviour.
Re:DeepSeek-Prover-V2
by simianwords on Wednesday April 30, 2025 @03:51PM
(#43849881)
i know. so why don't we have domain specific models as tools in consumer llm products
The point being that a separate expert model would be better at its own field than a single model that tries to be good at everything. Intuitively it makes sense, in practice I have seen anecdotes where finetuning a small model on domain data makes the model lose coherence on other topics.