Re:If you can't reproduce the model then it's not open-source
by RandomWorker on Wednesday January 17, 2024 @01:47PM
(#39031726)
There is a big effort being worked on in China, Yuanqing Lin gave an interview on the deep learning course that works on this magnitude [1]. They suggest that they will host both the resources to store the data, train the data, and have all those algorithms available in China.
Re:If you can't reproduce the model then it's not open-source
by tikhonj on Wednesday January 17, 2024 @03:47PM
(#39033223)
The public doesn't have the resources to train the largest state-of-the-art LLMs, but training useful LLMs seems doable. Maybe not for most individuals but certainly for a range of nonprofits, research teams and companies.
Re:If you can't reproduce the model then it's not open-source
by stcredzero on Wednesday January 17, 2024 @04:32PM
(#39033799)
Isn't is relatively easy for a smaller model to poke holes in the output of a larger model?
Re:If you can't reproduce the model then it's not open-source
by jncfhnb on Wednesday January 17, 2024 @05:14PM
(#39034312)
But not nearly as in reach as modifying open source models.
Re:If you can't reproduce the model then it's not open-source
by edgarvaldes on Wednesday January 17, 2024 @04:59PM
(#39034119)
Open Source and Free Software are not about the amount of data.
Can the public compete? What percentage of the technical public could we expect to participate, and how much data, compute, and data quality improvement could they bring to the table? I suspect that large corporations are at least an order of magnitude advantaged economically.