Federated Finetuning of OpenAI's Whisper on Raspberry Pi 5

Re:Federated Finetuning of OpenAI's Whisper on Raspberry Pi 5 by coder543 on Thursday November 16, 2023 @03:21PM (#38294921)

v3 only comes in one flavor: large.

I don’t think you’re going to have a good time running the large model on a Pi of any kind.

The large models are 32x slower than the tiny models, roughly.[0]

I just tested, and whisper.cpp on my Pi 4 can transcribe the 30-second a13.wav sample (“make samples” to fetch it) in 18.5 seconds.

You can do the math… 32x = 20 minutes transcribe 30 seconds of audio with the large model. Not a good time for most people.

The Pi 5 could be 2x to 3x faster.

[0]: https://github.com/openai/whisper/blob/main/README.md#availa...

[ Parent | on HN ]

Re:Federated Finetuning of OpenAI's Whisper on Raspberry Pi 5 by danieljanes on Thursday November 16, 2023 @03:24PM (#38294955)

I can confirm that we're seeing 2x to 3x faster (RPi 4 vs RPi 5) in some of our early tests

[ Parent | on HN ]

Re:Federated Finetuning of OpenAI's Whisper on Raspberry Pi 5 by jafermarq on Thursday November 16, 2023 @03:31PM (#38295037)

yes. Finetuning a whisper model on a RPi 5 is ~2x faster than on the RPi 4. Other stages involving data pre-processing with HF dataset is again 2x-3x faster.

[ Parent | on HN ]