Distil-Whisper: distilled version of Whisper that is 6 times faster, 49% smaller

Re:Distil-Whisper: distilled version of Whisper that is 6 ti... by rjwilmsi on Wednesday November 01, 2023 @10:02AM (#38098305)

That's the implication. If the distil models are same format as original openai models then the Distil models can be converted for faster-whisper use as per the conversion instructions on https://github.com/guillaumekln/faster-whisper/

So then we'll see whether we get the 6x model speedup on top of the stated 4x faster-whisper code speedup, at same/nearly same accuracy.

I would generally start with the assumption that if something is significantly faster the accuracy has to suffer a bit, but increasing model size and/or settings such as beam size to compensate should allow same accuracy and higher performance (just not all of the stated performance gain).

[ Parent | on HN ]

Re:Distil-Whisper: distilled version of Whisper that is 6 ti... by kkielhofner on Wednesday November 01, 2023 @12:58PM (#38101184)

Just a point of clarification - faster-whisper references it but ctranslate2[0] is what's really doing the magic here.

Ctranslate2 is a sleeper powerhouse project that enables a lot. They should be up front and center and get the credit they deserve.

[0] - https://github.com/OpenNMT/CTranslate2

[ Parent | on HN ]