News for hackers. Stuff that matters.
Re:LLaMA2 isn't “Open Source” – and why it doesn't matter by Der_Einzige on Friday July 21, 2023 @03:31PM (#36818441)
It's exactly the opposite. We have better ways to combine the knowledge of several models together than sampling them. (i.e. mixture of experts, model merges, etc) Relying on synthetic data from one LLM to train another LLM is in general a terrible idea and will lead to a race to the bottom.

There are no comments on this comment yet.