News for hackers. Stuff that matters.
Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by saboot on Monday September 05, 2022 @07:28AM (#32723133)
Is there a comparison with other fast cpu methods of matrix multiplication?
Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by corsix on Monday September 05, 2022 @07:41AM (#32723215)
Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by dougall on Monday September 05, 2022 @10:45AM (#32724872)
On M1, for single-precision, one AMX P-unit is ~1.64 TFLOPs, one P-core is ~102 GFLOPS. So ~16x core-for-core. But you have four P-cores for every AMX P-unit, so more like 4x. And for double-precision that shrinks to 2x (~410 GFLOPs to ~51GFLOPs).

(This is a simplification that doesn't include the E-cores, nor the AMX E-unit, but their contribution isn't huge. I suspect AMX throughput may have doubled on M2, but I haven't verified that.)

Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by Firadeoclus on Monday September 05, 2022 @11:34AM (#32725438)
That's a pretty huge amount of processing power hidden away! Are these experimentally confirmed performance numbers?

So this is 16x16 single precision fused multiply-adds at 3.2GHz with a throughput of one per clock, right? (16 x 16 x 2 x 3.2e9 = 1.6384e12) And does using fp16 quadruple throughput again? That would put AMX well above the GPU for fp16 matrix multiplication! (2.6 TFLOPs fp16/fp32 for 8-core GPU, 128 multiply-add per core at 1.278GHz)

How does it compare in terms of power, can it sustain 3.2GHz indefinitely or does it hit power/thermal limits fairly quickly?

Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by dougall on Monday September 05, 2022 @11:57AM (#32725758)
Correct, yep. These are theoretical numbers, measured in cycles from a P-core (with no loads/stores), real-world performance tends to be a little less (~93%): https://twitter.com/stephentyrone/status/1455665595677085697

FP16 only doubles throughput rather than quadrupling.

I haven't looked at power/thermals, so I can't really comment. (Though it's possible it's always running a bit under 3.2GHz, since I was measuring in clock cycles - that might be part of the 7% difference.)

Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by moonchild on Monday September 05, 2022 @07:33AM (#32723160)
IIRC I measured something like 10-50% performance difference (don't remember exactly, but it was somewhere in there), vs a reasonably well-regarded blas implementation. This was for dgemm specifically; I don't know if the story changes for smaller floats.
Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by saboot on Monday September 05, 2022 @07:41AM (#32723210)
Not too bad, I do scientific computing and choose Intel/Nvidia as their APIs for accelerated math operations are documented and supported for developers.

I've been paying attention to what Apple has been pushing with their M1/M2 chips, and I'm pretty tempted to try it out, but unless these features are documented and supported I can't feel comfortable writing programs relying on them.

Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by mhh__ on Monday September 05, 2022 @07:44AM (#32723230)
The API apple wants you to use is documented and presumably is here to stay.

That doesn't help if there's some edge case you'd need access to the raw ISA but still.

Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by Someone on Monday September 05, 2022 @07:55AM (#32723286)
Also, BLAS is part of that interface (https://developer.apple.com/documentation/accelerate/blas)

Of course it is a black box in that you can’t (realistically) try and speed it up. You still run the risk of Apple’s priorities being different from yours.

Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by kergonath on Monday September 05, 2022 @03:53PM (#32728625)
OTOH the interfaces are standard. Using Accelerate instead of the standard BLAS is just a compiler switch.
Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by kergonath on Monday September 05, 2022 @03:52PM (#32728611)
They are not supported directly, but you can ue them through the Accelerate framework, which has an optimised BLAS and FFT implementations (amongst many other things).
Re:Apple AMX instruction set (M1/M2 matrix coprocessor) by bfgoodrich on Monday September 05, 2022 @09:42AM (#32724198)
SGEMM / DGEMM using AMX2 (the first M1 has AMX2. The A14 has AMX1) is approximately 100% faster than the same running with NEON, which is already a specialized vector math system.