News for hackers. Stuff that matters.

A comment on Whisper: Nvidia RTX 4090 vs. M1 Pro with MLX. Parent comment

Re:Whisper: Nvidia RTX 4090 vs. M1 Pro with MLX by darkteflon on Wednesday December 13, 2023 @03:13PM (#38633541)
Yeah, I think everyone knows that Nvidia is doing a cracker job. But it is good to just be specific about these benchmarks because numbers get thrown around and it turns out people are testing different things. The other thing is that Apple is extracting this performance on a laptop, at ~1/8 the power draw of the desktop Nvidia card.

In any event, it’s super cool to see such huge leaps just in the past year on how easy it is to run this stuff locally. Certainly looking very promising.

[ Parent | on HN ]
Re:Whisper: Nvidia RTX 4090 vs. M1 Pro with MLX by modeless on Wednesday December 13, 2023 @03:17PM (#38633600)
The M2 Ultra that got the best numbers that I was comparing to is not in a laptop. Regardless, you're probably right that the power consumption is significantly lower per unit time. However, is it lower per unit work done? It would be interesting to see a benchmark optimized for power. Nvidia's power consumption can usually be significantly reduced without much performance cost.

Also, the price difference between a prebuilt 4090 PC and a M2 Ultra Mac Studio can buy a lot of kilowatt hours.

[ Parent | on HN ]