Bfloat16 support coming to Apple's Metal and PyTorch [video]

Re:Bfloat16 support coming to Apple's Metal and PyTorch [video] by dlewis1788 on Monday July 03, 2023 @12:42PM (#36575444)

Somehow missed this from WWDC23, but it looks like Sonoma will add support for bfloat16 with Metal, and there's an active PR to add support with the PyTorch MPS back-end (PR #99272). Since M2 added bfloat16 support at the hardware level, I'm assuming this will only be supported on M2 Macs.

That maxed out Mac Studio M2 w/ 192GB of memory now looks more appealing...

[ Parent | on HN ]

Re:Bfloat16 support coming to Apple's Metal and PyTorch [video] by kzrdude on Monday July 03, 2023 @02:04PM (#36576639)

Visible in the unofficial documentation for AMX instructions too - M2 only bf16 functionality - https://github.com/corsix/amx/blob/main/matfp.md

This matfp instruction computes an outer product and is a kernel for matrix multiplication.

[ Parent | on HN ]

Re:Bfloat16 support coming to Apple's Metal and PyTorch [video] by dlewis1788 on Monday July 03, 2023 @02:14PM (#36576804)

I didn't even know about Apple's AMX instructions until I clicked on your link. Very interesting - thanks!

[ Parent | on HN ]

Re:Bfloat16 support coming to Apple's Metal and PyTorch [video] by my123 on Monday July 03, 2023 @05:53PM (#36579304)

bf16 in Metal on macOS 14 is supported on all Macs. Emulated in software transparently.

[ Parent | on HN ]

Re:Bfloat16 support coming to Apple's Metal and PyTorch [video] by LoganDark on Monday July 03, 2023 @08:09PM (#36580514)

Yeah, Metal is pretty great because it runs the same on all Macs. Apple is really really good at this.

[ Parent | on HN ]