Re:Whisper: Nvidia RTX 4090 vs. M1 Pro with MLX
by WhitneyLand on Wednesday December 13, 2023 @02:32PM
(#38632976)
I’m afraid the article as well as your benchmarks can be misleading because there are a lot of different whisper implementations out there.
For ex ctranslate optimized whisper “implements a custom runtime that applies many performance optimization techniques such as weights quantization, layers fusion, batch reordering…”
Intuitively, I would agree with your conclusion about Apple’s M-Series being impressive for what they do but not generally competitive with Nvidia in ML.
Objectively however, I don’t see concluding much with what’s on offer here. Once you start changing libraries, kernels, transformer code, etc you end up with an apples to oranges comparison.
Re:Whisper: Nvidia RTX 4090 vs. M1 Pro with MLX
by modeless on Wednesday December 13, 2023 @02:34PM
(#38633018)
I think it's fair to compare the fastest available implementation on both platforms. I suspect that the MLX version can be optimized further. However, it will not close a 10x gap.
For ex ctranslate optimized whisper “implements a custom runtime that applies many performance optimization techniques such as weights quantization, layers fusion, batch reordering…”
Intuitively, I would agree with your conclusion about Apple’s M-Series being impressive for what they do but not generally competitive with Nvidia in ML.
Objectively however, I don’t see concluding much with what’s on offer here. Once you start changing libraries, kernels, transformer code, etc you end up with an apples to oranges comparison.