Re:Neural audio codecs: how to get audio into LLMs
by Workaccount2 on Tuesday October 21, 2025 @09:41AM
(#45655705)
Nothing is real understanding because we have no benchmark for understanding because we don't mechanistically know what understanding is. The best we have is people "vibe knowing" a benchmark that they made up on the spot.
Re:Neural audio codecs: how to get audio into LLMs
by vvolhejn on Wednesday October 22, 2025 @03:42AM
(#45665961)
There's a great blog post from Sander Dieleman about exactly this - why do we need a two step pipeline, in particular for images and audio?
https://sander.ai/2025/04/15/latents.html
For text, there are a few papers that train the tokenization and language model end-to-end, see: https://arxiv.org/abs/2305.07185
But I can say the same about tokenization. LLMs first convert groups of characters to tokens, then use that to generate tokens, and then convert the tokens back to characters. That's not real understanding! If LLMs are so smart, we should be able to skip the tokenization step.