Re:You can now train a 70B language model at home
by llmzero on Friday March 08, 2024 @06:53AM
(#39640235)
What is a little contradictory is that designing a system to use less resources can increases the number of people fine tuning models so that the final result can be a net global increase in the total energy use. A hypothetical goal could be to reuse fine tuning, that is designing a knowledge graph in which you fine tuning from a previously fine tuned model (like dynamic programming, save the result of previous computations). Lora allow us to store the small matrices with low cost.
[ Parent
| on HN ]
There are no comments on this comment yet.