Is there any other case where "open source" is used for something that can't be reproduced? Seems like a new term is required, in the concept of "open source, non-reproducible artifacts".

I suppose language changes. I just prefer it changes towards being more precise, not less.

This feels somewhat analogous to games like Quake being open-sourced though still needing the user to provide the original game data files.

But games like Quake are not "open source". They have been open-sourced, specifically the executable parts were, without the assets. This is usually spelled out clearly as the process happen.

In terms of functional role, if we're to compare the models to open-sourced games, then all that's been open-sourced is the trivial[0] bit of code that does the inference.

Maybe a more adequate comparison would be a SoC running a Linux kernel with a big NVidia or Qualcomm binary blob in the middle of it? Sure, the Linux kernel is open source, but we wouldn't call the SoC "open source", because all that makes it what it is (software-side) is hidden in a proprietary binary.

--

[0] - In the sense that there's not much of it, and it's possible to reproduce from papers.

No, the term is fine, “source” in “open source” refers to source code. A dataset by definition is not source code. Stop changing the meaning of words.

A dataset very much is the source code. It's the part that gets turned into the program through an automated process (training is equivalent to compilation).

Yes its "freeware" or any one of the similar terms we've used to refer to free software.

Academia - nowadays source is needed is a lot of conferences, but the datasets, depending on where/how it might have be obtained, just can't be used or not available and the exact results can't be reproduced.

Not sure if the code is required under an open source license, but it's the same issue.

---

IMO, source is source and can be used for other datasets. Dataset isn't available, bring your own.

In this case, the source is there. The output is there, and not technically required. What isn't available is the ability to confirm the output comes from that source. That's not required under open source though.

What's disingenuous is the output being called 'open source'.

In other words, it's everything except the one thing that actually matters.

Maybe, but it doesn't mean it's not open source.

The things that don't matter are, the thing that does isn't. Together, they can hardly be called open source.

The dataset is likely absolutely jam packed with copyrighted material that cannot be distributed.