Re:LLM Tech and a Lot More: Version 13.3 of Wolfram Language...
by ilaksh on Friday June 30, 2023 @08:49AM
(#36533925)
Relies on GPT-4. Maybe there could be a multistage way to get similar automated API usage with smaller open source models?
I wonder if it would be possible to train a 7B or 13B model to generate code in just one specific programming language. Train it with example problem input/ program output pairs. Then train another small model to translate natural language in a specific domain into an input for the coder model. And maybe a third to translate that into a different real programming language.
The point of this being that you can use smaller GPU instances and dedicate all of the limited power of each model to narrower domain that may be more tractable for it.
I wonder if it would be possible to train a 7B or 13B model to generate code in just one specific programming language. Train it with example problem input/ program output pairs. Then train another small model to translate natural language in a specific domain into an input for the coder model. And maybe a third to translate that into a different real programming language.
The point of this being that you can use smaller GPU instances and dedicate all of the limited power of each model to narrower domain that may be more tractable for it.