News for hackers. Stuff that matters.

A comment on Open Source AI Is the Path Forward. Parent comment

Re:Open Source AI Is the Path Forward by gowld on Tuesday July 23, 2024 @04:44PM (#41050610)
Can a human programmer edit the weights according to some semantics?
[ Parent | on HN ]
Re:Open Source AI Is the Path Forward by sebastiennight on Tuesday July 23, 2024 @04:52PM (#41050687)
It is possible to merge two fine-tunes of models from the same family by... wait for it... averaging or combining their weights[0].

I am still amazed that we can do that.

[0]: https://arxiv.org/abs/2212.09849

[ Parent | on HN ]
Re:Open Source AI Is the Path Forward by the8thbit on Thursday July 25, 2024 @04:58PM (#41073307)
This is absolutely wild.
[ Parent | on HN ]
Re:Open Source AI Is the Path Forward by sitkack on Tuesday July 23, 2024 @10:49PM (#41053197)
Yes, there is the concept of a "frakenmerge" and folks have also bolted on vision and audio models to LLMs.
[ Parent | on HN ]
Re:Open Source AI Is the Path Forward by root_axis on Tuesday July 23, 2024 @06:45PM (#41051802)
Yes. Using fine tuning.
[ Parent | on HN ]