Re:Federated Finetuning of OpenAI's Whisper on Raspberry Pi 5
by saqadri on Thursday November 16, 2023 @03:41PM
(#38295156)
This is cool. This might be a silly question, but what are the scenarios where it's useful for fine-tuning on the edge with small devices? I get inference on the edge, and curious about metrics on that for Whisper, but isn't it better to fine-tune on beefier infrastructure and then deploy it for inference on the edge?
[ Parent
| on HN ]
Re:Federated Finetuning of OpenAI's Whisper on Raspberry Pi 5
by danieljanes on Thursday November 16, 2023 @03:53PM
(#38295342)
The big opportunity on the edge is access to more data. Especially with the rise of end-to-end encryption, applications will be able to use more (and more diverse) data on the edge to get better model performance. It's generally true that training on beefier infrastructure is easier, but in the long run, nothing can beat access to better data. And edge hardware has gotten a lot faster over the last few years.
Re:Federated Finetuning of OpenAI's Whisper on Raspberry Pi 5
by triyambakam on Thursday November 16, 2023 @05:58PM
(#38296863)
It seems like one benefit of fine tuning on the edge is the data doesn't need to move around as much. My father taught me "don't move a pile of dirt twice", so maybe it is like that.