News for hackers. Stuff that matters.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by roughly on Monday June 05, 2023 @01:52PM (#36200160)
I’d been wondering how they were going to handle expandable memory with the M chip, since the integrated memory seemed pretty central to the design - seems like the answer is, “they’re not.” Be interested to see if PCI expansion is sufficient to satisfy the Max Pro market.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by bee_rider on Monday June 05, 2023 @02:17PM (#36200594)
Everything seems slightly parallel-universe in Apple world, looking at it from over here in x86/linux land.

I can see why they’d decide to go in-package with their memory, it is really very fast. And 192GB of memory is not a huge amount of memory in server/HPC land, but it is still a decent chunk of space. You could load up a Mac Pro with a bunch of PCIe nvme drives or something, I wonder if it would really be that hard to adapt to that.

I certainly wouldn’t turn down the chance to try, haha.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by AprilArcus on Monday June 05, 2023 @04:47PM (#36203366)
NVMe wouldn't give you the best latency, but a 16x PCIe card loaded with DRAM and addressable as a scratch disk doesn't sound bananas. I wonder why Apple didn't market something like that as a first party solution.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by nickpeterson on Monday June 05, 2023 @08:49PM (#36206620)
I expect that expansion cards from Apple are going to be the way they differentiate the mac pro line. I'm betting that Apple is going to come out with discrete GPU's based on their integrated graphics and sell them as add-ons for the Pros for ML/AI workloads and likely to help with normal stuff like rendering in Autodesk/Adobe/Finalcut type apps.

That said, it isn't the bet I'd make if I was tasked with purchasing for a company. I'd get the studios in a reasonably high config and then sell them every 2-3 years and replace with newer model. You'd pass the pro's in perf pretty readily from CPU bumps alone and likely recoupe a lot of your investment.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by bee_rider on Monday June 05, 2023 @11:34PM (#36207723)
A RAM disk formatted as swap? Probably would work. Funny concept.

Let’s lie to OS and tell it that this memory is actually a disk. Then, it will lie to the programs, and tell them that that disk is actually memory.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by adanto6840 on Monday June 05, 2023 @08:36PM (#36206510)
I've got a PCIe RAID card loaded with NVMe SSDs on my 2019 MP. It's really great. Unfortunately, the latest macOS versions are not amenable to using other drives as a boot drive (IIRC may be related to secure enclave stuff?). Lots of (third party) macOS apps don't play real well with additional storage devices either, and the OS itself has some (relatively minor) hurdles to deal with, too. For project/data/"working directory"/general filesystem usage, it's great.

That said, up to now it's probably provided me with ~ fastest possible drive speeds available for macOS. For business/dev purposes, I really can't complain (except about price & lack of modern Nvidia support - but alas), even though I'm already using M1 Max these days - the single-core speed is just too good.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by tiffanyh on Monday June 05, 2023 @02:32PM (#36200861)
Mac Pro = Mac Studio + Expansion Cards

I have to imagine this will be a huge disappointment to some, because 192GB of shared memory is way less than the 1.5TB of RAM available on the "old" Mac Pro.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by peoplearepeople on Monday June 05, 2023 @02:57PM (#36201359)
Perhaps someone will come out with a new PCIe card with a load of RAM slots on it, and then writes a kernel driver to map the pcie card pages to appear as regular pages
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by Kon-Peki on Monday June 05, 2023 @04:11PM (#36202861)
That's an interesting idea. But do you actually need to go all the way to making the extra memory appear as a contiguous part of the system memory? I am thinking about CUDA unified memory and perhaps some parallels to your idea.

The number of applications that are likely to use the extra memory is probably pretty small. So if you have some sort of framework that those developers can integrate into their software, you've probably done everything you need to do.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by PlutoIsAPlanet on Monday June 05, 2023 @06:41PM (#36205155)
You could use them as some kind of swap or ram disk, but I don't believe as normal RAM would be possible due to how CPUs work.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by daemontus on Tuesday June 06, 2023 @02:27AM (#36208900)
Someone already has :) It's called CXL. Both Intel and AMD are diving head first into it for servers. Apple would just need to support it...
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by pwthornton on Monday June 05, 2023 @02:08PM (#36200437)
The approach they took is certainly enough to satisfy the video and related markets. It won't help those who need truly staggering amounts of ram for their workloads. It may not be a big use case for the Intel Mac Pro, but it a use case nonetheless.

For video editing, color grading, audio editing, 3D animation, etc. this new machine seems really strong. I am not sure if there is anything beyond that, however.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by WWLink on Monday June 05, 2023 @05:14PM (#36203703)
I think that was deliberate. I am kinda amused at how they keep on narrowing the scope of what the mac pro is intended to be used for lol.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by gumby on Monday June 05, 2023 @05:26PM (#36203931)
They probably have very good data on how their machines are used, and are optimizing for the fat part of the market.

This may not be a great machine for training models, which is what I happen care about (I couldn't care less about video). I wonder how big the model generation market actually is though.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by rodgerd on Monday June 05, 2023 @09:28PM (#36206890)
I do wonder in these conversations how many people are actual users of these things - I'm not, I got a Studio last year to manage my photography addition better, and the Pro is pointless for that - but you'd assume that Apple have talked to major Pro users, and they've said things like "oh, no for training a model I just spin up terabyte instances on AWS, why would I do that on a workstation?", or whatever.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by gumby on Tuesday June 06, 2023 @11:50AM (#36214685)
I assume the audience is the Disneys/Pixars of the world, and the people making ads, etc in 8K. They presumably need that I/O. For them the absurdly expensive monitor is a rounding error, and the extra local disk is worth it over running over a SAN. Perhaps also some computationally adjacent applications like CFD or weather prediction.

I struggle to think of anyone else with such an application. There are applications that could benefit from the features but it wouldn’t be cost effective (e.g. something more systolic where it’s better to rent cloud service, even if it were nominally slower, because once you’re done you don’t have the hardware lying around).

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by USB5 on Monday June 05, 2023 @06:07PM (#36204657)
>3D animation

Disagree. All the good GPU-based rendering engines need CUDA, and none of them are optimized for Apple silicon. Octane (the one in the demo) is trash, only good for fancy titles and that sort of thing.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by haldean on Monday June 05, 2023 @08:20PM (#36206363)
Octane, Redshift, and eevee/Cycles all work on Metal, and Renderman and Mantra are CPU-only for now -- RM/Mantra have beta-versions that are GPU enabled, and Karma is nvidia-only and RM XPU doesn't work on macs at all yet, but I don't think many people are using either in production very much yet. I think the only GPU renderer people are using in production pipelines that's really locked to NVIDIA chips is Arnold, right?
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by USB5 on Monday June 05, 2023 @08:29PM (#36206447)
I can't speak for cycles, but Redshift is very slow on OpenCL. I've never heard of using it on Metal. I know it was written for CUDA and didn't even support OpenCL for a long time. Do you know how it runs on Metal?

I don't think V-Ray GPU will run on anything but Nvidia, and if it will then it's definitely slow.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by haldean on Monday June 05, 2023 @08:35PM (#36206499)
Ah yeah forgot about V-Ray. I haven't tried Redshift on Metal, nope; all my work has been Renderman and Cycles recently. Curious to hear how well it works, people online seem to say it's pretty fast but hard to tell marketing copy from actual performance when it comes to renderers.

FWIW Cycles is definitely slower on my M2 than it was on my 3080 but it's not a huge difference -- maybe 20% slower? I still have to let the render run overnight either way haha.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by USB5 on Monday June 05, 2023 @11:16PM (#36207614)
Interesting. Which M2 machine are you using?
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by haldean on Tuesday June 06, 2023 @01:10AM (#36208361)
Mac Mini with M2 Pro
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by kllrnohj on Monday June 05, 2023 @02:44PM (#36201076)
The integrated memory design doesn't prevent doing it over a dimm slot. It seems more they just didn't want to deal with a ddr5 or some bespoke connector.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by qwytw on Monday June 05, 2023 @04:41PM (#36203291)
Also allowing that would lower Apples margin and their computers would remain usable for much longer which would result in even less profits...

The technical issues are totally insignificant compared to this.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by sliken on Monday June 05, 2023 @02:48PM (#36201153)
192GB ram does seem like enough for a large fraction of the potential market, especially @ 800GB/sec which makes it much more usable than similar amounts of ram on Intel/AMD desktops at 1/8th the bandwidth.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by StillBored on Monday June 05, 2023 @04:27PM (#36203088)
Except that my fairly modest upper midrange desktop has ~1.2TB/sec of memory bandwidth...

Cause, I just added the GPU and system RAM bandwidth numbers together. Which is what needs to be kept in mind with much of this. Yes that is a lot of memory bandwidth and its hella useful for some subset of users, but its shared, and largely pointless for a lot of CPU bound tasks. But OTOH, may not be enough for many GPU bound ones.

It also assumes that pretty much every other CPU manufacture on the planet are idiots for optimizing for latency and putting in large caches to compensate (aka the desktop parts from AMD/intel have only _two_ channels, vs the 8+ in the server/workstation parts) and price discriminating for the parts that have more CPU bandwidth. AKA, you can get amd machines in the same ballpark (or possibly faster depending on how fast you can get 24 channels of DDR5 to run).

So, I'm not saying which is better because its likely workload dependent, but to claim its a blanket insurmountable advantage is questionable. Particularly since the price ranges we are talking about a similar machine is probably a 64 core threadripper plus a fat nvidia GPU or four and the shear core count and raw GPU compute is probably a win in most workloads.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by USB5 on Monday June 05, 2023 @05:38PM (#36204162)
I just built a workstation with a 32 core Threadripper Pro, 128 GB RAM, and an RTX 4090 for $6,500.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by sliken on Monday June 05, 2023 @06:45PM (#36205210)
Likely better on any workloads that are GPU heavy and fit in 24GB of vram.

Two data points from Apple:

  a) M2 Ultra 24 cores, 128GB ram, 2TB storage = $5,200
  b) M2 ultra 24 cores, 192GB ram, 4GB storage = $6,600
Likely 1/4th the size, 1/4th the power consumption, and 4x the ram bandwidth. Have you by chance played with any LLMs? Just saw a post that someone managed 5 tokens/sec with the llama 65B model.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by USB5 on Monday June 05, 2023 @07:17PM (#36205670)
I don't care about playing with LLMs, so this is simply meaningless to me.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by kamranjon on Monday June 05, 2023 @11:54PM (#36207888)
But for many people training or running inference on large models is important and the apple approach actually makes that feasible in a desktop solution which is pretty awesome.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by nightski on Monday June 05, 2023 @08:40PM (#36206534)
Weird I priced out the Mac Pro M2 Ultra 192GB just now on the site and it came out to $9,599.00 (with only a 1TB SSD).
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by sliken on Monday June 05, 2023 @06:50PM (#36205295)
Sure, but what if a normal C code needs more bandwidth?

Or if a GPU code needs more than 12-16GB of memory (normal cards) or 24GB (if you get a 4090)?

What I like about the apple approach is that low end laptops/desktops get 100GB/sec. Pay another $500 get 200GB/sec. Pay another $500 get 400GB/sec. Pay another $1000 get 800GB/sec and still fits in a small desktop. On the PC side with AMD/Intel you get the same memory bandwidth for the low, medium, and high end chips. Until you upgrade to a threadripper, which is a 280 watt chip, on an expensive motherboard, usually in a rather large PC case and makes the mac studio look cheap.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by kcb on Monday June 05, 2023 @11:02PM (#36207536)
> Sure, but what if a normal C code needs more bandwidth?

https://www.anandtech.com/show/17024/apple-m1-max-performanc...

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by wtallis on Tuesday June 06, 2023 @12:27AM (#36208076)
Putting things in context, the 3-thread measured memory bandwidth measured there on the M1 Max is approximately equal to the maximum theoretical memory bandwidth available across all channels and chiplets of a current Ryzen Threadripper PRO. If the M1 Ultra doubles the achievable memory bandwidth by virtue of having twice as many CPU clusters, then an 8-thread test should still be able to match AMD's latest EPYC processors with 12-channel DDR5.

(I don't know how well AMD's current processors do with utilizing the socket's full DRAM bandwidth from a limited number of chiplets, but I wouldn't be surprised if it's a more severe limitation than what M1 Max/Ultra show with their CPU clusters. It looks like only the 12-chiplet EPYC processors actually use all the links from the IO die to the CPU chiplets.)

So the inability to use all the DRAM bandwidth from the CPU cores, while perhaps disappointing, isn't exactly a weakness for Apple's processors compared to the competition.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by sliken on Tuesday June 06, 2023 @11:58AM (#36214786)
I believe the chiplet links are pretty generous, they handle L3 <-> chiplet traffice and chiplet <-> chiplet traffic. On the smaller configs they use 2 links per chiplet.

Thinks like McCalpin do not seem to show much difference on the different number of chiplet Epycs, although I've not personally tested the newest Genoa chips.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by adgjlsfhk1 on Monday June 05, 2023 @04:31PM (#36203154)
This is competing with servers though, and Epyc 9004 series has 460 GB/sec (and up to 6TB of ram per socket). Apple still gives you faster connection, but I feel like servers below 256GB ram are pretty rare these days.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by 221qqwe on Monday June 05, 2023 @04:38PM (#36203262)
> much more usable than similar amounts of ram

People keep repeating this but how does higher bandwidth (probably not 8x higher though) compensate for a lower amount of RAM?

It's not quite as silly as the people saying that 8GB in the base config 'feels' much faster than 8GB on a PC cause the drive/swap are "so fast" but still..

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by lambdasquirrel on Monday June 05, 2023 @05:41PM (#36204222)
Maybe it adds another step to the memory mountain? Folks who use these kinds of workstations might think of in-package RAM as just the next level of cache, if someone goes ahead and makes a card with comparatively slower memory card slots.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by dathinab on Monday June 05, 2023 @07:08PM (#36205551)
The DDR5 bandwidth is ~51.2 GiB/s per "channel".

With consumer desktop CPUs having 2 "channels" (~102.4GiB/s).

And prosumer desktop/workstation CPUs (e.g. Threadripper) having 4 "channels" (~204.8GiB/s).

While apple is claiming 819.2GiB/s.

That is _max throughput is 4x/8x more_ (depending if you compare it to prosumer (fair comparison) or consumer (unfair comparison for a $7k system) hardware.

Now the _max_ part is important, Apple mainly reaches this by having more bandwidth, i.e. parallelism.

Mainly (oversimplified) the per "CPU memory channel" bandwidth for x86 is 64bit (for 51.2GiB/s) so the M1 Ultra is roughly comparable to having 16 "CPU memory channels" instead of 8 (prosumer) or 4 (consumer).

Some applications can take advantage of this nicely and will scale potentially even to 4x/8x speed, most probably will not some might even have neglible improvements. But applications which use a lot of RAM (as much as they can get) and most of the RAM they use is "warm" (e.g. doesn't just lie around with little access but isn't super "hot", i.e. highly contented either) will profit quite a lot.

On the other hand applications which use little RAM but the same small RAM region very heavily contented likely will hardly at all profit and will likely run faster on overclocked RAM on consumer systems.

Luckily for apple most the typical use case they sell their pro desktop model for belong more/mostly in the first category.

Additionally if you can run Linux on this system they might become _very_ interesting for some scientific applications for some users. I mean even e.g. Zen 4 EPYC CPU only have 12 "CPU memory channels" not 16 and it's much easier to put a desktop box "somewhere" then a server unit.

Side note: I say "CPU channel" in quotes because while it tends to be the marketing term things are more complicated in practice, e.g. in general DDR5 is splitting the 64bit channels into 32bit sub-channels, and just listing the channel width and throughput is still not painting the whole picture at all, e.g. the latency also matters for some applications (hence why OC can make sense) etc.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by rektide on Monday June 05, 2023 @11:18PM (#36207624)
AMD's Strix Halo is an APU targeted next year that supposedly has 256-bit wide LPDDR5X memory, on laptops I think, which should be nice.

I do hope folks other than Apple can get fast! The new CAMM dual channel module will hopefully help reduce footprint but there's plenty of soldered down systems so it's not really required. It also surprises me there's not a GDDR based APU, except I guess too many games need both a bunch of system ram and video ram and 32GB vram is expensive and has significant lower draw. Apple going wide is really the obvious move, & doing it on package was the best way to do it, it seems.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by dathinab on Monday June 05, 2023 @11:34PM (#36207724)
> not a GDDR based APU

AFIK Apple currently doesn't have a foodhold on gaming outside of phone/tablet games (where they are strong, but the games are used to LPDDR perf.).

And while many of the Graphic applications people do use would profit from it I'm not sure it's that big of benefit.

In the end a lot of users just do daily tasks on Apple laptops and for that battery life absolutely trumps any speed benefits GDDR gives.

I guess the desktop versions could/should use GDDR, but there are two issue: 1. Heat, 2. more design/variance in the CPU production supply chain.

The 2nd can drive up cost by quite a surprising high amount.

The 1st point I'm not supper sure about. But AFIK one big problem with things which are stacked on-die is heat as all the heat of the CPU needs to go through whatever is stacked onto it to reach the cooler. And while there are probably all kinds of tricks to improve this I could imagine that using a GDDR which has a higher power draw and produces more heat itself could make this more of an issue. But that is purely speculative.

I guess we have to see until around 2025+ to know if such stacked chips are more prone to die a early (i.e. <5 years) heat death. Especially some of the Air models without heat pipes could be at risk if used in a less climate controlled environment. Or it could be all perfectly fine. I'm looking forward to finding out.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by rektide on Tuesday June 06, 2023 @12:10AM (#36207974)
Apple already outclasses everyone in throughput & throughput/watt especially. In case it wasn't already crystal clear, I was talking about everyone else finding ways to get their throughput up to more competitive levels, such as by supporting GDDR.
Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by dathinab on Tuesday June 06, 2023 @01:43AM (#36208609)
But a huge part of their throughput numbers is them having the equivalent of 16 memory channels.

AFIK it's not that e.g. AMD couldn't do that and use on-die LPDDR5, I mean that is a bit different but not sustainable harder then using their chiplet + X3D tech.

The problem is that for AMD it's not a good business decision. First they need to be price competitive (a problem Apple doesn't have). Most applications outside some areas scale much more with the speed/latency of RAM then with bandwidth increase (beyond some basic level). In turn for a lot of e.g. desktop Ryzen processor use cases which are not media processing there is not enough value into adding many more channels. Especially if you consider that for large parts of the prosumer/server space IT admins will be really unhappy with on-die RAM Apple can afford forcing it, AMD can not. The reason is that while for desktop systems RAM death is rarely an issue in the server/heavily used workstation spectrum RAM death is not uncommon. Even for media processing or multi VM servers going beyond a certain number of channels (less then 16) is unlikely to be a good financial decision. I would go as far as arguing if the M2 Ultra wouldn't be based on tightly gluing 2 processors together and that processors have 8 channels because they are sold with a focus for media processing it wouldn't have anywhere close to 16 channels (but for Apple the cost of having less then 16 channels with their design is higher then the cost of having them, partially because they also don't sell server they don't want to compete with accidentally and similar).

Where I'm going with this is that outside of some media processing targeted products they don't need to "get their throughput up to more competive level" and getting it to having a some additional 8 channel choices (e.g. for some high end laptops) would likely be good enough in practice. And in turn we are unlikely to see more. And in turn they have no reason to try tricks like using GDDR memory.

The way Apple can push hardware vendors here is less because of a need for many more channels, but because for a want.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by rektide on Tuesday June 06, 2023 @02:38AM (#36208984)
I don't know but I could see the possibility that maybe AMD really can use more throughput for mobile chips.

To spitball some figures, the rx7600 is 13b transistors, 165watts, and runs off 128-bit 288GBps GDDR6 memory. Or take 2017's rx580, but which was 256-bit GDDR5 good for 224GBps, at similar power.

An APU is going to be considerably lower power than either of these discrete gpus. I think I somewhat overestimated AMD's ability to scale up their APUs to be throughput limited, in most cases. The 256GBps LPDDR5X memory they're planning should offer a nice bump over where we are be pretty good.

I'm a bit surprised to see the memory bandwidth not being as constraining a factor as I had first guessed.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by dathinab on Tuesday June 06, 2023 @03:39AM (#36209481)
I mean it is a constraining factor for live on-the fly real time multi 8k-track video editing and similar ;=)

It's just that most people don't do that on a daily usage basis.

Maybe not a I need 800GiB/s constraining factor but definitely a I don't want just 200GiB/s constraining factor AFIK.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by sliken on Tuesday June 06, 2023 @11:49AM (#36214683)
> The problem is that for AMD it's not a good business decision.

The PS5 and XboxX have an AMD APU (CPU+IGPU) with a wide memory interface. Seems like a fine decision. What surprises me is they haven't brought it to low/medium range desktops, until Strix Halo in 2024.

Re:Apple unveils new Mac Studio and brings Apple Silicon to ... by rodgerd on Monday June 05, 2023 @09:39PM (#36206978)
> People keep repeating this but how does higher bandwidth (probably not 8x higher though) compensate for a lower amount of RAM?

I guess it depends on your use case, but back when part of my day job was debugging performance problems with JVM-hosted applications, one of the things that was most noticeable was that the degree to which latency blows up memory use - whether the latency was GC, disk, network, DB queries, whatever. It all turns into holding items in memory longer before they get processed (which in the case of older JVMs, turns into a death spiral of GC thrashing, which blows up processing times further, until your application is staggering along).

Increasing memory can alleviate this, but only up to a point - and it can make things worse, because you've now got significant overhead managing the in-flight workloads.

It's also possibly that this is simply a decision driven by what Apple can produce with their M2 chips at this point, and that they wanted to offer 384 or 640 GB as the maximum, and everyone is making excuses for them.