Did you reply to the right comment? Feels like we’re talking about different things altogether.
Re:Apple introduces M4 chip
by bingbingbing777 on Tuesday May 07, 2024 @07:22PM
(#40292662)
What AI is Apple scaling?
Re:Apple introduces M4 chip
by bionhoward on Tuesday May 07, 2024 @08:51PM
(#40293245)
Seen MLX folks post on X about nice results running local LLMs. https://github.com/ml-explore/mlx
Also, Siri, and consider: you’re scaling AI on apple’s hardware, too, you can develop your own local custom AI on it, there’s more memory available for linear algebra in a maxed out MBP than the biggest GPUs you can buy.
They scale the VRAM capacity with unified memory and that plus a ton of software is enough to make the Apple stuff plenty competitive with the corresponding NVIDIA stuff for the specific task of running big AI models locally.
> there’s more memory available for linear algebra in a maxed out MBP than the biggest GPUs you can buy.
But this hardly applies to 95% if not more people of all people running Apple's hardware, the fastest CPU/GPU isn't worth much if you can fit any at least marginally useful LLM model on the 8GB (or less on iPhones/iPads) of memory that you device has?
Apple is selling hardware and scaling AI by utilizing it is simply a smart move.
Instead of building huge GPU clusters, having to deal with NVIDIA for GOUs (Apple kicked NVIDIA out years ago because of disagreements), Apple is building mainly on existing hardware.
This is in other terms utilizing CPU power.
On the other hand this helps their marketing keeping high price points when Apple now is going to differentiate their COU power and therefore hardware prices over AI functionality correlating with CPU power. This is also consistent with Apple stopping the MHz comparisons years ago.