Neural audio codecs: how to get audio into LLMs

Re:Neural audio codecs: how to get audio into LLMs by orena on Tuesday October 21, 2025 @11:06PM (#45664511)

How many epochs did you train with ? 100k hours is not a lot for an LLM, Feels like bitter lesson

[ Parent | on HN ]

Re:Neural audio codecs: how to get audio into LLMs by vvolhejn on Wednesday October 22, 2025 @03:39AM (#45665933)

I train for 1M steps (batch size 64, block size 2048), which is enough for the model to more-or-less converge.

It's also a tiny model for LLM standards, with 150M parameters. The goal wasn't really to reach state of the art but to show how the performance of a single language model architecture can be vastly different when you just change the tokenizer.

[ Parent | on HN ]

Re:Neural audio codecs: how to get audio into LLMs by singularfutur on Wednesday October 22, 2025 @03:58AM (#45666063)

To get around state of the art, how many parameters would be needed with your approach?

[ Parent | on HN ]