News for hackers. Stuff that matters.
Re:You can now train a 70B language model at home by llmzero on Friday March 08, 2024 @06:53AM (#39640235)
What is a little contradictory is that designing a system to use less resources can increases the number of people fine tuning models so that the final result can be a net global increase in the total energy use. A hypothetical goal could be to reuse fine tuning, that is designing a knowledge graph in which you fine tuning from a previously fine tuned model (like dynamic programming, save the result of previous computations). Lora allow us to store the small matrices with low cost.

There are no comments on this comment yet.