Cool experiment. It will be amazing if this can match the WER of the best open weight models at better speed.

People aren't paying enough attention to how great EmbeddingGemma 2 and Gemma 4 are.

I am experimenting with the reader with more varied audio. Plus, possibly an ultra fast post processing layer.

can this tool transcribe latin language texts? McWhisper allegedly does it but it outputs gibberish

Yay for Latin. Care to share your project?

Thanks for putting this out there!

can you turn this into a decision model

Isn't EmbeddingGemma2 already a decision model? Am I misunderstanding your question?

it was a joke

EmbeddingGemma 2 is not natively a decision model; it is a multimodal embedding model

What would a decision model add to and audio transcriber?

Theoretically - if the change was deeper than just restricting output - you could feed it thousands worth of audio tokens and get a structured response in milliseconds rather than minutes?

Say classify recorded interviews into viable/not viable candidate?

Reducing cost for a large scale corporation call center. Press or say one.

Basic ML is probably even cheaper. But LLMs might have higher accuracy. “Uhh one I think” = 1

I haven't tested audio, but in general, for specific tasks, classifiers tend to do better than decision models ("general classifiers" based on LLMs): https://jeffyclassify.com/blog/when-llms-fail-at-classificat...

It can already classify decently well, and I tried training it as a decision model, It worked well on domains I trained it for. It was beating Liquid's small decision model, but somehow, it never generalized. Out of the trained domains, it was just above chance.

You can train a traditional classifier on it quite well.

But the one-shot and few-shot classification performance of decision models really needs in-context learning. The most common way of getting that is by grafting a small LLM onto it, which is certainly possible.

At that point though, why not just use a small LLM with audio capabilities. Gemma E2B is a good model that is already setup for this, and training a decision model LoRA adapter is very easy.

Edit: Apparently people have got decision models working on EmbeddingGemma, eg: https://huggingface.co/mrm8488/embeddinggemma-2-ft-typed-dec.... I'm not seeing it ranked on https://huggingface.co/spaces/multimodalart/jev-decision-ind... so I have questions about how well it does in-context learning.

It was your post yesterday that made me aware of granite-speech-5.0-470m-turboctc (which I guess you used as a comparator because it's already very fast).

Thank you!

  uv tool install mlx-audio
  mlx_audio.stt.generate \
  --model ibm-granite/granite-speech-5.0-470m-turboctc \
  --audio.wav \
  --output-path transcript

(Until now I've been using mlx-community/parakeet-tdt-0.6b-v3)

Yes. Granite is really fast. IBM has actually been releasing some great models that are flying under the radar