Re:If you can't reproduce the model then it's not open-source
by CHY872 on Wednesday January 17, 2024 @04:53PM
(#39034044)
And, that's obviously fun, because with LLMs, you have the LLM itself which cost hundreds of thousands in compute to train, but given you have the weights it's eminently fine-tunable. So it's actually not really like Linux - rather it's closer to something like a car, where you had no hope of making it in the first place but now you have it, maybe you can modify it.
[ Parent
| on HN ]
Re:If you can't reproduce the model then it's not open-source
by patmorgan23 on Wednesday January 17, 2024 @05:57PM
(#39034796)
So in this case, the weights are the source code and the training material + compute time is like the software development process that went into creating the source code.
It would probably take well over a million dollars in engineering hours to recreate the postgres source code from scratch, just as it would take millions in compute to rebuild the weights.