Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by Zambyte on Wednesday April 24, 2024 @03:43PM
(#40148693)
Training data is a build dependency for the weights. You cannot realistically get the same weights without the same training data.
Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by ko27 on Wednesday April 24, 2024 @04:07PM
(#40148944)
Developer's mindset, knowledge and tooling is also a build dependency for any open source code. You can not realistically get the same code without it.
Re:Snowflake Arctic Instruct (128x3B MoE), largest open sour...
by Zambyte on Wednesday April 24, 2024 @07:00PM
(#40150926)
> you can not realistically get the same code without it
You mean the same source code? Because... I agree. That's why it's important for the source to be open. Both in the context of software and language models.
No, the same way training is not a build dependency for the weights source code. You can literally compile and run them without any training data.