You can use it to improve knowledge in indirect ways:
- get the model better at crafting queries for an external data source
- get the model better a tool usage to do computation with an external system
- get more useful embeddings from BERT/SBERT
- tell the model what it cannot answer accurately
But in general, fine tuning is noise right now because 99% of the people chasing it actually don't need it.
If you want to change how the model presents text, use fine tuning. If you want to change what the model can present, fine tuning is a hopeless way of doing it.
If this is what I want, then what is the right way to do it?
If you're answering questions on current events, hook it up to a news API and let it take in a user query, return a search to you, and give the result back to the LLM before it responds.
Right now LLMs perform best for factual information if you treat them like flexible glue between the user and your data sources. Any knowledge you leverage from the LLM itself should "static" knowledge that can hold true across all user queries.
For example, you can use the LLM's knowledge of marketing to produce marketing copy via chain of thought. Trends in marketing copy can change, but the actual act of making marketing copy takes a static set of knowledge.
_
Don't use the LLM to surface competitors for the product, because one person will ask about 'Coke' and it'll say 'Pepsi', and another will name their founded in 2023 company and it'll hallucinate: especially if you're asking for structured output
Instead let the LLM form some searches, take the results and let the LLM use chain of thought on what defines a competitor, and surface that result
For example, letting the LLM return a result to the user and return a list of entities it wants to save details on for later, then letting it query past entries allows long term memory for a class of entities.
Over all it will be a very long time before approaches like that are beat by fine tuning, because as the models improve, this strategy gets stronger: the model gets better are crafting queries, you can provide more context in the context window.
Meanwhile fine tuning in knowledge will get more difficult as MOE and other unforseen developments reset what works as far as directly tinkering with weights.
If so, let say in the future, we have an LLM with 100K token context windows but with a subsystem where it notices some knowledge keeps being repeated in the context and then store that knowledge for finetuning when the LLM is not doing inference. Basically a mirror of the way we human work? Is that possible? An LLM that constantly improved and can adapt to new knowledge?