News for hackers. Stuff that matters.
Re:Open source ChatGPT with LLaMA implementation trains 15x ... by visarga on Monday February 27, 2023 @11:43AM (#34958583)
You can already do that. Take your task first to GPT-3 and collect a bunch of outputs. Then fine-tune a small model on them. Works well, but you need to extend the dataset to cover all edge cases because the small model can't draw on the vast knowledge GPT-3 has.

There are no comments on this comment yet.