Re:OpenAI is too cheap to beat
by nico on Thursday October 12, 2023 @02:59PM
(#37861501)
> I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedback
This makes a lot of sense. A small model that “knows” enough English and a couple of programming languages should be enough for it to replace something like copilot, or use plug-ins or do RAG on a substantially larger dataset
The issue right now is that to get a model that can do those things, the current algorithms still need massive amounts of data, way more than what the final user needs
This makes a lot of sense. A small model that “knows” enough English and a couple of programming languages should be enough for it to replace something like copilot, or use plug-ins or do RAG on a substantially larger dataset
The issue right now is that to get a model that can do those things, the current algorithms still need massive amounts of data, way more than what the final user needs