Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by Zambyte on Wednesday April 24, 2024 @12:59PM
(#40146919)
They don't have a weird definition of open source. I recently outlined a LLM chat that I think clearly outlines this: https://news.ycombinator.com/item?id=40035688
[ Parent
| on HN ]
Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by ko27 on Wednesday April 24, 2024 @03:02PM
(#40148206)
A bunch of code was autocompleted or generated by IDEs, are open source developers supposed to release the source code of that IDE to be OSI compliant?
Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by Zambyte on Wednesday April 24, 2024 @03:11PM
(#40148306)
Is the IDE a primary input for building the program? Is the IDE a build dependency? Probably not. Certainly not based on the situation you described.
The LLM equivalent here would be programmatically generating synthetic input or cleaning input for training. You don't need the tools used to generate or clean the data in order to train the model, and thus they can be propriety in the context of an open source model, so long as the source for the model is open (the training data).
Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by ko27 on Wednesday April 24, 2024 @03:38PM
(#40148645)
> Is the IDE a primary input for building the program? Is the IDE a build dependency?
No, the same way training is not a build dependency for the weights source code. You can literally compile and run them without any training data.
Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by Zambyte on Wednesday April 24, 2024 @03:43PM
(#40148693)
Training data is a build dependency for the weights. You cannot realistically get the same weights without the same training data.
Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by ko27 on Wednesday April 24, 2024 @04:07PM
(#40148944)
Developer's mindset, knowledge and tooling is also a build dependency for any open source code. You can not realistically get the same code without it.
Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by Zambyte on Wednesday April 24, 2024 @07:00PM
(#40150926)
> you can not realistically get the same code without it
You mean the same source code? Because... I agree. That's why it's important for the source to be open. Both in the context of software and language models.