By this definition almost any binary can be "open source" since hex editors exist. (Or more usefully, you can use ghidra et al. to do more interesting changes.) I know GPL has a very specific view of things, but I'd like to quote an excerpt that I think is generally applicable from https://www.gnu.org/licenses/gpl-3.0.html -
> The “source code” for a work means the preferred form of the work for making modifications to it. “Object code” means any non-source form of a work.
Which is why I'm fine with "open weights", because that's saying the object code is under an open license.
> The problem with requiring "build from scratch" for open source models is that the number of interesting models with training data that can be openly licensed is close to zero.
So? If the number of open source models is zero, then the number of open source models is zero.
It is legal to train on copyrighted materials, provided they were obtained legally. Most companies also train their models using user interactions with previous iterations.
It is impossible to release this data publicly, let alone license it to a third party. However, I believe that at least the training code and the data processing pipeline could, and should, be released in order to claim a model is truly "open source."
That said, Allen AI actually released several models with the full datasets available. It is impressive how they pushed the models' performance despite training on a limited set of publicly available data. Kudos to them.
The problem with requiring "build from scratch" for open source models is that the number of interesting models with training data that can be openly licensed is close to zero.
If you trained your model on an unlicensed scrape of the web you can't release the data under an open source license!
The Open Source Initiative have a bunch of their thinking around this in their FAQ for the "Open Source AI definition": https://opensource.org/ai/faq#isn-t-training-data-required-t...