News for hackers. Stuff that matters.
Re:Hertz-dev, the first open-source base model for conversat... by a2128 on Monday November 04, 2024 @03:41AM (#42039684)
Moshi never released the base model, only two conversationally finetuned models. They also never released training code except for the codec. Though I don't see any training code for Hertz either, just 3 inference notebooks, and model code full of no_grad. No paper either to help me understand how this was trained and what the architecture is like. So I'm not too sure about researcher-friendliness unless I'm missing something.
Re:Hertz-dev, the first open-source base model for conversat... by nicholas-cc on Monday November 04, 2024 @04:10AM (#42039828)
We're working on a HuggingFace release that will help with finetuning. We'd like to do a paper, after a larger release - we're a team of 4.
Re:Hertz-dev, the first open-source base model for conversat... by netdevnet on Monday November 04, 2024 @04:25AM (#42039892)
Very impressive for just 4 people. What's the team background and how long have you been working on this?
Re:Hertz-dev, the first open-source base model for conversat... by unit149 on Monday November 04, 2024 @04:43AM (#42039985)
For a rag-tag group of transcendental audiophiles operating electronic circuitry, it ionizes and atomizes well.
Re:Hertz-dev, the first open-source base model for conversat... by programjames on Monday November 04, 2024 @01:13PM (#42044405)
I'm not part of their team, but lived with them for a couple months. They've been working on it for ~5 months, and their background is 16-20 year olds who are too smart for university.