Why would I even care about that for ML?
>I know the usual Apple bashers
Apple fanboys found one benchmarks which looks fine (Performance per watt) and decided it applies to everything and the only thing that ever matters - even when it's totally irrelevant.
It used to be they cared about Mhz, until it became 'the Mhz myth' when Apple was losing.
Erm .... because you are buying an ML LAPTOP ?
If you're going to be buying a laptop to leave it plugged into the mains the whole time when doing intensive tasks, then you might as well buy a workstation ?
The whole point is you can buy an Apple silicon laptop and you can use it without performance loss as ... shock, horror ... a portable device.
Well, this is how most people use laptops, isn't it? The main benefit of portability is that you can move to another location in between the tasks, with little to no set up and teardown, and without having to suspend or power off the machine. Few people do anything on their laptops while walking around - the usual use case is taking your computer to a meeting room or a cafe, and the first order of business for anyone is usually to find a power outlet.
No, it is not how most people use laptops.
For example, airline travel.
Let's start at the airport first, shall we ?
Most airports, for "security reasons" don't have many power sockets hanging around. And if they do, they are generally at inconvenient centralised charging desks. So that's a bit of a pain if you want to work somewhere reasonably quiet and away from the hoards.
Of course you may be an Airline Elite member, in which case you get access to the lounges. But lounges are generally busy places. Many don't have that many power sockets.You also need to wrestle with power-adapters etc. Finally, your laptop charging cord is a trip-hazard waiting to happen.
Then let's get on board the aircraft.
The majority of short-haul aircraft do not have any sort of charging mechanism available, so that's that ruled out.
Meanwhile long-haul wide-body aircraft will generally have power (and more modern ones USB, although the USB power generally will be limited and insufficient for a hungry laptop).
So overall, in reality, the only time you'll be able to comfortably use your laptop on mains power during air travel is on a long-haul wide-body flight where you are seated for 14+ hours (other than toilet breaks etc.).
> the first order of business for anyone is usually to find a power outlet
The whole point is that Apple makes that "first order of business" a deprecated relic of the past.
With an Apple silicon laptop you can turn up and work most or all of the day without having to concern yourself where the nearest power outlet is.
I'd rather it plugged into the mains so I can also plug it into a nice big monitor and a large keyboard and a mouse, so I can use it like a human being and not a computer appendage.
A portable device is for BYOD. Actually working with it while on the go? I'd never do this unless it's an emergency.
I suspect this will gradually change, perhaps especially now a lot of effort has been made to bring tooling such as PyTorch over to Apple silicon.
> on CPU instead of GPU
But Apple isn't doing it on CPU.
You are thinking in terms of x86 discrete components.
Apple Silicon is a fully integrated architecture including unified memory. That's what makes it so efficient.
I think it’s just a question of time until Apple offers hardware support for BFLOAT and other formats on the GPU and AMX (they already have BFLOAT16 in the CPU), at which point their ML performance will improve dramatically.
At 1/10th of the speed...
Yawn.
As has already been clearly explained numerous times in this discussion, e.g. @ribit[1] below, there is no throttling on Apple Silicon when running on battery.
... yet. The computational demand of typical work will increase to eat up the extra capacity, as it always does - and the only way for a laptop to handle the same workload plugged or unplugged is to cap its plugged-in performance.
Aah, that famous marketing "up to" ! So I guess that's in certain cherry-picked scenarios then ?
Also what they don't tell you is that the Apple will happily do ML for hours on end ON BATTERY with zero performance impact. Whilst an x86 laptop will need to be permanently tethered to your nearest power socket in order to obtain top performance. Otherwise you will suffer a performance drop and kill off your battery life.
I know the usual Apple bashers gonna hate, but fact of life its that for most people Apple have done a sterling job on their Apple Silicon laptops. No doubt more to come with the M2 in due course.