News for hackers. Stuff that matters.
Re:2022 Mac Studio (20-core M1 Ultra) Review by danieldk on Thursday April 28, 2022 @03:28PM (#31197102)
Above, we’re looking at parallel performance of the NASA USM3D CFD solver as it computes flow over a classic NACA 0012 airfoil section at low speed conditions.

If this solver relies on matrix multiplication and uses the macOS Accelerate framework, you are seeing this speedup because M1 Macs have AMX matrix multiplication co-processors. In single precision GEMM, the M1 is faster than an 8 core Ryzen 3700X and a bit slower than a 12 core Ryzen 5900X. The M1 Pro doubles the GFLOPS of the M1 (due to having AMX co-processors for both performance core clusters). And the M1 Ultra again doubles the GFLOPs (4 performance core clusters, each with an AMX unit).

Single-precision matrix multiplication benchmark results for the Ryzen 3700X/3900X and Apple M1/M1 Pro/M1 Ultra are here:

https://twitter.com/danieldekok/status/1511348597215961093?s...

Re:2022 Mac Studio (20-core M1 Ultra) Review by JohnTHaller on Thursday April 28, 2022 @09:44PM (#31200425)
The 5900X can do 24 threads, so why artificially limit it to 16 threads? Unless that's because the M1 Ultra has 16 performance cores. Why include the 3 year old 3700X? Why not include the 32 thread 5950X which can be had in a complete system for 1/3 the price of the Mac Studio M1 Ultra?
Re:2022 Mac Studio (20-core M1 Ultra) Review by simonh on Friday April 29, 2022 @02:48AM (#31202143)
This isn’t a scientific study, it’s one guy who reviews the pro Macs he gets to work on. Also he’s been using the same benchmark for years and the figures for previous Mac Pros are historical, so past tests weren’t conducted with any knowledge of the M1 architecture.
Re:2022 Mac Studio (20-core M1 Ultra) Review by threeseed on Thursday April 28, 2022 @09:56PM (#31200490)
They clearly just picked 16 threads because it is enough to demonstrate the point.

M1 only has 4 performance cores and they still ran it with 16 threads. So it's not like they are trying to skew the benchmark by not showing the potential of the 5900x.

Also the 5950x is 2 years old and still is an active product.

Re:2022 Mac Studio (20-core M1 Ultra) Review by danieldk on Friday April 29, 2022 @06:15AM (#31203405)
The performance generally regresses when using more threads on the Ryzen CPUs.

Also, maybe I can only benchmark on the hardware that I actually have?

Re:2022 Mac Studio (20-core M1 Ultra) Review by hedgehog on Thursday April 28, 2022 @10:07PM (#31200570)
The 5900X is a fine reference as a common chip. The peak performance should be at 12 threads, it has 12 cores, and that's what the table appears to show.
Re:2022 Mac Studio (20-core M1 Ultra) Review by sharken on Friday April 29, 2022 @01:32AM (#31201787)
It would make more sense to compare systems with similar price points.

In this case that would mean ThreadRipper systems and not Ryzen.

Re:2022 Mac Studio (20-core M1 Ultra) Review by danieldk on Friday April 29, 2022 @06:18AM (#31203422)
Sure, if I had a ThreadRipper, I’d be happy to add the results. I don’t have an M1 Ultra either, but someone was kind enough to do the benchmark for me.
Re:2022 Mac Studio (20-core M1 Ultra) Review by bigiain on Friday April 29, 2022 @04:59AM (#31202980)
For the guy who, you know, actually did the comparison here... Of _course_ it makes most sense to compare the new system he has on his desk with the previous systems he's already done relevant performance tests as comparison points.

If you want a comparison between a Threadripper system and an M1 Ultra system, I'd be fascinated to read your blog post with your results.

Re:2022 Mac Studio (20-core M1 Ultra) Review by snapcore on Thursday April 28, 2022 @07:53PM (#31199685)
The scaling curve is different from the other cpus which implies the performance bottleneck is not in matrix multiplication (which would be nearly linear on the older cpus because it is embarrassingly parallel).
Re:2022 Mac Studio (20-core M1 Ultra) Review by paulmd on Thursday April 28, 2022 @08:18PM (#31199849)
CFD is highly memory-bandwidth-bottlenecked, it is in fact pretty much the prototypical memory-bandwidth-bottlenecked task.

The performance scaling you see between systems pretty much corresponds to the memory bandwidth in those configurations.

Note that on the M1, the CPU can only access a fraction (about 25% iirc) of the total memory bandwidth, you have to use the GPU to really get the full performance of the M1 here.

Re:2022 Mac Studio (20-core M1 Ultra) Review by sliken on Thursday April 28, 2022 @08:31PM (#31199943)
I saw this quantified (I think at anandtech), something like 220GB/sec out of 400GB/sec on the M1 max. So something north of 50%.

Also keep in mind that normal x86-64's, even without an IGP only get about 60-65% of peak, even with nothing else sharing the memory bus. I often see this quantified with McCalpin's stream benchmark.

So the M1 Ultra likely has a pretty impressive memory bandwidth of around 440GB/sec, which isn't a large fraction of 800GB/sec, but it still more than any other desktop or server chip I know of. The AMD Epcy maxes out at 8 channels of DDR-3200, which is in the neighborhood of 208GB/sec peak, with an observed bandwidth of 110-120GB/sec.

Re:2022 Mac Studio (20-core M1 Ultra) Review by GeekyBear on Thursday April 28, 2022 @09:24PM (#31200294)
> I saw this quantified (I think at anandtech)

Correct. The numbers we have are from their M1 Max deep dive, with the M1 Ultra being two M1 Max chips fused together.

For CPU cores:

>Adding a third thread there’s a bit of an imbalance across the clusters, DRAM bandwidth goes to 204GB/s, but a fourth thread lands us at 224GB/s and this appears to be the limit on the SoC fabric that the CPUs are able to achieve, as adding additional cores and threads beyond this point does not increase the bandwidth to DRAM at all. It’s only when the E-cores, which are in their own cluster, are added in, when the bandwidth is able to jump up again, to a maximum of 243GB/s.

https://www.anandtech.com/show/17024/apple-m1-max-performanc...

GPU cores:

>I haven’t seen the GPU use more than 90GB/s (measured via system performance counters). While I’m sure there’s some productivity workload out there where the GPU is able to stretch its legs, we haven’t been able to identify them yet.

Other:

>That leaves everything else which is on the SoC, media engine, NPU, and just workloads that would simply stress all parts of the chip at the same time. The new media engine on the M1 Pro and Max are now able to decode and encode ProRes RAW formats, the above clip is a 5K 12bit sample with a bitrate of 1.59Gbps, and the M1 Max is not only able to play it back in real-time, it’s able to do it at multiple times the speed, with seamless immediate seeking. Doing the same thing on my 5900X machine results in single-digit frames. The SoC DRAM bandwidth while seeking around was at around 40-50GB/s – I imagine that workloads that stress CPU, GPU, media engines all at the same time would be able to take advantage of the full system memory bandwidth, and allow the M1 Max to stretch its legs and differentiate itself more from the M1 Pro and other systems.

Re:2022 Mac Studio (20-core M1 Ultra) Review by AceJohnny2 on Thursday April 28, 2022 @08:51PM (#31200079)
> of around 440GB/sec, which isn't a large fraction of 800GB/sec

Where's the 800GB/s from?

Re:2022 Mac Studio (20-core M1 Ultra) Review by sliken on Thursday April 28, 2022 @08:59PM (#31200127)
Peak = never observed, but calculated from clock speed * bus width. Much like the speed of light, you'll never see it.

That number for the M1 Ultra (from the OP's post) = 800GB/sec. McCalpin's stream benchmark is often cited as a practical/useful number for usable bandwidth using a straight forward implementation in C or Fortran without trying to play games, much like the vast majority of codes out there.

Also note that the x86-64's in the world use a strict memory model that results in a lower fraction of observed bandwidth vs peak. Arm has a looser memory model which achieves a higher fraction of peak.

Re:2022 Mac Studio (20-core M1 Ultra) Review by rodgerd on Thursday April 28, 2022 @10:03PM (#31200539)
> memory-bandwidth-bottlenecked task.

The thing that's interesting for me around this is: I hate the on-die memory. I hate the idea of not being able to upgrade after your order, or a few years down the road. It has both practical problems and offends my inner nerd.

But.

This is a useful example of why there's value in it. It seems unlikely that you'd get as good a result with the traditional memory architecture.

Re:2022 Mac Studio (20-core M1 Ultra) Review by KennyBlanken on Friday April 29, 2022 @01:36AM (#31201803)
People who need this sort of system aren't going to be looking to expand memory capacity in "a few years down the road." They'll be looking to purchase or lease the current model.

Also, if you really need more memory - buy the config with more memory and sell the old config. Resale on macs is usually superb.

I can't remember hearing a friend or gaming buddy say "I upgraded my ram." I haven't upgraded ram in any machine I've owned in twenty years and even before that, it was rare. There was never any point in putting that sort of money into an out of date CPU and memory architecture.

If you owned a trashcan Mac Pro and were a working creative, would you be upgrading its memory this month? Nope...

Re:2022 Mac Studio (20-core M1 Ultra) Review by FullyFunctional on Friday April 29, 2022 @08:44PM (#31212273)
I contest this claim. I'm professional and I know I'm not alone in buying an iMac 5K with 16 GiB or less explicitly to later upgrade to much more with 3rd party (like Crucial) memory for vastly less than Apple asks.

That said, if the memory is fixed like on the Ultra, then sure, we'll buy what we need (and a bit more).

Re:2022 Mac Studio (20-core M1 Ultra) Review by greedo on Friday April 29, 2022 @09:03AM (#31204551)
I think what people are really looking for is expandable memory so that you can buy a base version, and then add cheaper 3rd party RAM.
Re:2022 Mac Studio (20-core M1 Ultra) Review by simondotau on Thursday April 28, 2022 @10:49PM (#31200819)
I’m hoping that the Mac Pro introduces a two tier memory architecture so you can get the best of both worlds.
Re:2022 Mac Studio (20-core M1 Ultra) Review by bee_rider on Thursday April 28, 2022 @03:40PM (#31197250)
Just for anyone as out of touch with the MacOS ecosystem as me: Accelerate includes a BLAS implementation, so at least seems plausible (depending on how this library was compiled) that their special instructions might have been used.
Re:2022 Mac Studio (20-core M1 Ultra) Review by stephencanon on Thursday April 28, 2022 @05:35PM (#31198504)
If it were taking advantage of Accelerate, the performance would be much higher, but also the scaling would be quite different. Look at the scaling in the tweet you linked--it's anything but linear in the number of cores used.
Re:2022 Mac Studio (20-core M1 Ultra) Review by staticfloat on Thursday April 28, 2022 @08:08PM (#31199787)
This linear scaling per-core doesn't match my experience with using the AMX co-processor. In my experience, on the M1 and the M1 Pro, there are a limited number of AMX co-processors that is independent of the number of cores within the M1 chip. I wrote an SO answer exploring some of the performance implications of this [0], and since I wrote that another more knowledgable poster has added more information [1]. One of the key takeaways is that appears to be one AMX coprocessor per "complex", leading us to hypothesize that the M1 Pro contains 2 AMX co-processors.

This is supported by taking the code found in the gist [2] linked from my SO answer and running it on my M1 Pro. Compiling it, we get `dgesv_accelerate` which uses Accelerate to solve a medium-size linear algebra problem, that typically takes ~8s to finish on my M1 Pro. While running, `htop` reports that the process is pegging two cores (analogous to the result in my original SO answer on the M1 pegging one core; this supports the idea that the M1 Pro contains two AMX co-processors). If we run two `dgesv_accelerate` processes in parallel, we see that they take ~15 seconds to finish. So there is some small speedup, but it's very small. And if we run four processes in parallel, we see that they take ~32 seconds to finish.

All in all, the kind of linear scaling shown in the article doesn't map well to the limited number of AMX co-processors available in Apple hardware, as we would expect the M1 Max to contain maybe 8 co-processors at most. This means we should see parallelism step up in 8 steps, rather than 20 steps as was shown in the graph.

Everything I just said is true assuming that a single processor, running well-optimized code can completely saturate an AMX co-processor. That is consistent with the tests that I've run, and I'm assuming that the CFD solver he's running is well-written and making good use of the hardware (it does seem to be doing so from the shape of his graphs!). If this were not the case, one could argue that increasing the number of threads could allow multiple threads to more effectively share the underlying AMX coprocessor and we could get the kind of scaling seen in the article. However, in my experiments, I have found that Accelerate very nicely saturates the AMX resources and there is none left over for future sharing (as shown in the dgesv examples).

Finally, as a last note on performance, we have found that using OpenBLAS to run numerical workloads directly on the Performance cores (and not using the AMX instructions at all) is competitive on larger linear algebra workloads. So it's not too crazy to assume that these results are independent of the AMX's abilities!

[0] https://stackoverflow.com/a/67590869/230778 [1] https://stackoverflow.com/a/69459361/230778 [2] https://gist.github.com/staticfloat/2ca67593a92f77b1568c03ea...

Re:2022 Mac Studio (20-core M1 Ultra) Review by torginus on Thursday April 28, 2022 @05:02PM (#31198172)
But that makes me think, what prevents people from running these calculations on the GPU? Even the memory is shared - the few 100 gflops they get out of the M1 ultra is pocket change in GPU terms.
Re:2022 Mac Studio (20-core M1 Ultra) Review by my123 on Thursday April 28, 2022 @07:20PM (#31199437)
Apple GPUs don't support FP64 at all - and Apple only provides Metal as a (supported and not deprecated) API.

NVIDIA GPUs have much more ubiquitous programming models, from CUDA C++ to OpenMP (both C++ and Fortran), allowing them to be much more useful.

AMD ROCm (a CUDA clone, but not managed too well...) isn't in a great state today, but it definitely provides an OpenMP compiler, for C++ only though, Fortran is not covered.

Intel GPUs have a much better SW story on all fronts for GPGPU than AMD's, and OpenMP is supported for both C++ and Fortran there.

tldr: if you want to run this code on GPUs, the two vendors to look for are NVIDIA and Intel. The others just don't have the software stack to do so. (without more heavyweight porting)

Of course, optimising for GPUs also adds work. That said, an OpenMP version is a very good starting point.

Re:2022 Mac Studio (20-core M1 Ultra) Review by paulmd on Thursday April 28, 2022 @08:14PM (#31199821)
You can always emulate FP64 - it is not performant to do so, but since CFD is essentially memory-bandwidth-bottlenecked rather than limited by shader performance, this probably is not a big concern. You've got shaders sitting around waiting anyway.

The M1 architecture only allows the CPU to access a fraction of the memory bus, it is around 25% of the total bandwidth or so iirc. The rest has to go to the GPU or NPU or one of the other units, so even if your FP64 code was super shitty emulated with no hardware support, on paper you should be able to quadruple performance by utilizing it.

If you can "pre-digest" it at all, and then push it back from the GPU to the CPU, that might be another viable approach.

Of course none of this will exist out of the box in some 90s FORTRAN code!

Re:2022 Mac Studio (20-core M1 Ultra) Review by my123 on Thursday April 28, 2022 @08:20PM (#31199868)
> You can always emulate FP64

That's what's done by Intel for their newer consumer GPUs AFAIK, zero FP64 units present on die, but an emulation mode can be enabled to expose that support. (which is then done by software on the GPU)

> it is around 25%

50%, ~ 200GB/sec on an M1 Max out of ~ 400GB/sec

Re:2022 Mac Studio (20-core M1 Ultra) Review by the_svd_doctor on Thursday April 28, 2022 @05:05PM (#31198219)
Well that particular NASA code is from the 90’s and in Fortran. Maybe with MPI. That’s probably part of the reason why.
Re:2022 Mac Studio (20-core M1 Ultra) Review by willis936 on Thursday April 28, 2022 @07:36PM (#31199564)
Don't all high performance math libraries have the option of LAPACK interfaces?
Re:2022 Mac Studio (20-core M1 Ultra) Review by whatever1 on Friday April 29, 2022 @02:18AM (#31202007)
Because GPUs calculations are the same type of calculations. If you have a problem where you need to add plus 1 in all of the elements of a matrix, then GPUs are great. Games are a great example because you can take one pixel and calculate its values over time without much worry of what happens around it. Similar story for neural nets.

For CFD it is a different problem. In each time-step you essentially solving a linear system Ax=b, so even if you are looking at element x(100,100,timeperiod2) you need to also know the value of the element x(1,1,timeperiod1).

There are some algorithms that decouple the problem by introducing residuals for each element and then trying to iteratively reduce them, but as you can see, it is not a linear increase in speed as someone would had expected by looking at the GPU specs.

TLDR: Yes you can do CFD with GPUs, but dont expect miracles.

Re:2022 Mac Studio (20-core M1 Ultra) Review by dekhn on Friday April 29, 2022 @10:03AM (#31205226)
using GPUs for CFD is pretty standard these days and has been for a few years. GPUs are fantastic for solving linear systems.

Nobody uses GPUs for "linear speedups by looking at GPU specs". That's not how accelerators are measured.

Re:2022 Mac Studio (20-core M1 Ultra) Review by whatever1 on Friday April 29, 2022 @11:14AM (#31206203)
"using GPUs for CFD is pretty standard these days and has been for a few years. "

It is not standard in OpenFoam (only though rapidCFD) or Ansys (only this year they made an announcement for full acceleration).

"GPUs are fantastic for solving linear systems."

*Some* linear systems. They are horrible for sparse algebra.

Re:2022 Mac Studio (20-core M1 Ultra) Review by CamperBob2 on Thursday April 28, 2022 @06:36PM (#31199091)
If a CPU doesn't run the OS and software I need to run, it might as well be a GPU or a Cray 9 or something dug out of the wreckage at Area 51. All of which are good descriptions of a CPU available only on the increasingly-proprietary Mac platform.

So this entire thread is kind of pointless with regard to many real-world use cases.

Re:2022 Mac Studio (20-core M1 Ultra) Review by alimov on Thursday April 28, 2022 @07:54PM (#31199691)
Saying nothing in so many words.

> “If a CPU doesn't run the OS and software I need to run, it might as well be a GPU or …”

> So this entire thread is kind of pointless with regard to many real-world use cases.

Does this kind of reasoning apply to everything that doesn’t suit your unique personal needs?

Personally I think this is an interesting thread, but I don’t use the qualifier of “is it useful to me” to determine whether something is interesting.

Re:2022 Mac Studio (20-core M1 Ultra) Review by gigatexal on Friday April 29, 2022 @05:40AM (#31203223)
I know you're getting downvoted but I feel more or less the same. That's why I am following the Asahi linux folks so closely. I really want them to succeed and for their work to make it possible for Fedora or Ubuntu to be a seamless experience on the M-* Apple hardware.
Re:2022 Mac Studio (20-core M1 Ultra) Review by lupire on Thursday April 28, 2022 @08:11PM (#31199804)
Alternatively, your OS of choice may as well be a Cray 9 OS if it doesn't run on modern high-end hardware.

Philosophical purity rarely provides top performance. And that's OK, slow and steady works for me, because I'm not doing max performance computing.

Re:2022 Mac Studio (20-core M1 Ultra) Review by madengr on Thursday April 28, 2022 @07:25PM (#31199486)
Came here to say the same. I don’t know of any computational heavy engineering software available for the Mac. Unfortunately I’m stuck on Windows for all my software.
Re:2022 Mac Studio (20-core M1 Ultra) Review by _ph_ on Friday April 29, 2022 @03:33AM (#31202418)
Well, the Apple Silicon Macs just arrived. If they are a compelling architecture for engineering software, then sooner or later a company will support them. Also, most Unix-based software should be rather easy portable, especially programs for computations which do not depend on a GUI. There is a lot of engineering software which is only available on Unix and these days mostly on Linux, as it comes from times when Windows and PCs where not even remotely a consideration as a platform.
Re:2022 Mac Studio (20-core M1 Ultra) Review by lupire on Thursday April 28, 2022 @08:07PM (#31199778)
That's good info but omg why a tweet of a crop of screenshot of a document.