News for hackers. Stuff that matters.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by StillBored on Monday June 05, 2023 @04:27PM (#36203088)
Except that my fairly modest upper midrange desktop has ~1.2TB/sec of memory bandwidth...

Cause, I just added the GPU and system RAM bandwidth numbers together. Which is what needs to be kept in mind with much of this. Yes that is a lot of memory bandwidth and its hella useful for some subset of users, but its shared, and largely pointless for a lot of CPU bound tasks. But OTOH, may not be enough for many GPU bound ones.

It also assumes that pretty much every other CPU manufacture on the planet are idiots for optimizing for latency and putting in large caches to compensate (aka the desktop parts from AMD/intel have only _two_ channels, vs the 8+ in the server/workstation parts) and price discriminating for the parts that have more CPU bandwidth. AKA, you can get amd machines in the same ballpark (or possibly faster depending on how fast you can get 24 channels of DDR5 to run).

So, I'm not saying which is better because its likely workload dependent, but to claim its a blanket insurmountable advantage is questionable. Particularly since the price ranges we are talking about a similar machine is probably a 64 core threadripper plus a fat nvidia GPU or four and the shear core count and raw GPU compute is probably a win in most workloads.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by USB5 on Monday June 05, 2023 @05:38PM (#36204162)
I just built a workstation with a 32 core Threadripper Pro, 128 GB RAM, and an RTX 4090 for $6,500.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by sliken on Monday June 05, 2023 @06:45PM (#36205210)
Likely better on any workloads that are GPU heavy and fit in 24GB of vram.

Two data points from Apple:

  a) M2 Ultra 24 cores, 128GB ram, 2TB storage = $5,200
  b) M2 ultra 24 cores, 192GB ram, 4GB storage = $6,600
Likely 1/4th the size, 1/4th the power consumption, and 4x the ram bandwidth. Have you by chance played with any LLMs? Just saw a post that someone managed 5 tokens/sec with the llama 65B model.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by USB5 on Monday June 05, 2023 @07:17PM (#36205670)
I don't care about playing with LLMs, so this is simply meaningless to me.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by kamranjon on Monday June 05, 2023 @11:54PM (#36207888)
But for many people training or running inference on large models is important and the apple approach actually makes that feasible in a desktop solution which is pretty awesome.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by nightski on Monday June 05, 2023 @08:40PM (#36206534)
Weird I priced out the Mac Pro M2 Ultra 192GB just now on the site and it came out to $9,599.00 (with only a 1TB SSD).
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by sliken on Monday June 05, 2023 @06:50PM (#36205295)
Sure, but what if a normal C code needs more bandwidth?

Or if a GPU code needs more than 12-16GB of memory (normal cards) or 24GB (if you get a 4090)?

What I like about the apple approach is that low end laptops/desktops get 100GB/sec. Pay another $500 get 200GB/sec. Pay another $500 get 400GB/sec. Pay another $1000 get 800GB/sec and still fits in a small desktop. On the PC side with AMD/Intel you get the same memory bandwidth for the low, medium, and high end chips. Until you upgrade to a threadripper, which is a 280 watt chip, on an expensive motherboard, usually in a rather large PC case and makes the mac studio look cheap.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by kcb on Monday June 05, 2023 @11:02PM (#36207536)
> Sure, but what if a normal C code needs more bandwidth?

https://www.anandtech.com/show/17024/apple-m1-max-performanc...

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by wtallis on Tuesday June 06, 2023 @12:27AM (#36208076)
Putting things in context, the 3-thread measured memory bandwidth measured there on the M1 Max is approximately equal to the maximum theoretical memory bandwidth available across all channels and chiplets of a current Ryzen Threadripper PRO. If the M1 Ultra doubles the achievable memory bandwidth by virtue of having twice as many CPU clusters, then an 8-thread test should still be able to match AMD's latest EPYC processors with 12-channel DDR5.

(I don't know how well AMD's current processors do with utilizing the socket's full DRAM bandwidth from a limited number of chiplets, but I wouldn't be surprised if it's a more severe limitation than what M1 Max/Ultra show with their CPU clusters. It looks like only the 12-chiplet EPYC processors actually use all the links from the IO die to the CPU chiplets.)

So the inability to use all the DRAM bandwidth from the CPU cores, while perhaps disappointing, isn't exactly a weakness for Apple's processors compared to the competition.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by sliken on Tuesday June 06, 2023 @11:58AM (#36214786)
I believe the chiplet links are pretty generous, they handle L3 <-> chiplet traffice and chiplet <-> chiplet traffic. On the smaller configs they use 2 links per chiplet.

Thinks like McCalpin do not seem to show much difference on the different number of chiplet Epycs, although I've not personally tested the newest Genoa chips.