Uhm, we can expect close to 8 FP32 TFLOPS from the CPUs alone on the M3 Ultra. It comes with 4 tensor engines (AMX) each capable of about 2 TFLOPs.

M3 Max GPU benchmarks around 14 TFLOPs, so the Ultra should score around 28 TFLOPs.

Double the numbers for FP16.