Back when that M1 MAX vs 3090 blog post was released, I ran those same tests on the M1 Pro (16GB), Google Colab Pro, and free GPUs (RTX4000, RTX5000) on the Paperspace Pro plan.

To make a long story short, I don't think buying any M1 chip make senses if your primary purpose is Deep Learning. If you are just learning or playing around with DL, Colab Pro and the M1 Max provide similar performance. But Colab Pro is ~$10/month, and upgrading any laptop to M1-Max is at least $600.

The "free" RTX5000 on Paperspace Pro (~$8 month) is much faster (especially with fp16 and XLA) than M1 Max and Colab Pro, albeit the RTX5000 isn't always available. The free RTX4000 is also a faster than M1 Max, albeit you need to use smaller batch sizes due to 8GB of VRAM.

If you assume that M1-Ultra doubles the performance of M1-Max in similar fashion to how the M1-Max seems to double the gpu performance of the M1-Pro, it still doesn't make sense from a cost perspective. If you are a serious DL practitioner, putting that money towards cloud resources or a 3090 makes a lot more sense than buying the M1-Ultra.

Apple Silicon (including base M1) actually has great FP16 support at the hardware level, including conversions. So it is wrong to say it only supports FP32.

> The Core ML runtime dynamically partitions the network graph into sections for the Apple Neural Engine (ANE), GPU, and CPU, and each unit executes its section of the network using its native type to maximize its performance and the model’s overall performance. The GPU and ANE use float 16 precision, and the CPU uses float 32.

Also, this exploration (https://tlkh.dev/benchmarking-the-apple-m1-max#heading-neura...) reports the 5.1-5.3 TFLOPS FP16 ballpark performance.

No one plays games on a Mac.

And it has nothing to do with GPU performance but rather the fact that the audience simply isn't interested in gaming on it and so there is no economic incentive to target them.

So the GPU performance that matters to Mac users and is relevant to Apple is not games but rather content creation, production etc.

My wife does (I play games on Linux).

I have friends who own Macs who reluctantly dual boot to Windows just to play some games -- they would completely ditch Windows if they could just play every game on Mac.

I see there are Mac games on Steam.

All of this points to the situation being more nuanced than "no one plays games on Mac".

According to the steam hardware survey, Windows is 95%, Macos is 4%, and Linux is 1%. And to dig deeper you'd need to see what games that 5% of the non-Windows is playing - are they simpler games that don't need graphics acceleration (e.g. puzzle games, roguelikes, etc) or ones that do?

My desktop is an Intel NUC running Ubuntu, and yeah I play games on it. Slay the Spire, Spacechem, even some older MMOs like DDO or LoTRO (which run but at 15-20 fps since that system just has Intel Iris). I'm unable to even start many others (e.g. Grim Dawn) due to not having dedicated graphics.

So yeah it's nuanced but lots of games that need a graphics card don't run or even display on that system.

That's why I have a windows gaming system too. I'm realistic, the market just isn't there. I used to have a Mac (dropped it in 2018) and if I still did I'd subscribe to Geforce Now or just do console gaming.

Who says Mac users aren't gamers? It's a self perpetuating vicious cycle: gamers use Windows because that's where the majority of games are, so developers keep targeting Windows, so gamers use Windows, and so ad infinitum.

But people who use Macs enjoy games as well. They would rather not dual boot to Windows.

It's not true that Mac users don't play games. Rather, it's that most games are on Windows, which is a shame.

What I think you mean is competitive gamers and AAA gamers do not play on Apple hardware. This is mostly true today ofc, but keep in mind that's actually not the majority of the gaming market. Apple is raking it in from its gaming market.

I have no idea what the answer is. Personally I have run a number of games on macOS, via Steam, and via Boot Camp and virtualization. Some popular MMOs like Final Fantasy XIV, World of Warcraft, and Eve Online have macOS clients, though Guild Wars 2 discontinued theirs.

Apple Arcade apparently has enough Mac users that Apple has a reason to support it on macOS as well as iOS.

And Apple apparently has some reason to support iOS/iPadOS games on Apple Silicon Macs as well (though it could just be a side effect of a future iOS-macOS merger or hybrid device.)

...and how many would play more games on their Macs if they were available?

I refuse to believe Mac users are less fond of videogames. Based on my personal observation of the Mac users I know, they enjoy games as much as anyone.

At least casual games on iTunes for the iPad have a vast library, with many genuine good games (source: me, my iPad 2 was mainly a gaming platform for me, never found other uses for it).

I do realize more complex games are a different beast. But casual? I'd say Apple fans love them.

Video content creation specifically is where it mostly achieves what the graphs indicate, and that's mostly a "well yeah, video decoder ASICs are really efficient. I'll take things we already knew for $100, Alex"

If the software is optimized, the graphics hold up fine.

Won't even get into that it can't run most things you'd want that kind of hardware for.

edit: Of course the M1 max only shipped in laptops, so... who knows.

I’d assume that’s what most of the chonk is about, no?

> Based on the thermal and power consumption characteristics of previous chips I would not be surprised if say ~120W is the max power draw of this thing.

The Max could be brought up to 90W or so.

If you think this is still a problem, you haven't used any recent Macs. The current MB Air and MB Pro both run very cool even under prolonged heavy loads.

Apple's management of any and all heat issues has been far better than any competitors for a while now.

Only if you define "for a while" as "since a year ago with the introduction of the M1".

Apple refused to make a thicker laptop or one with better ventilation to adequately cool the CPUs & GPUs they were sticking in them. They were among if not the worst of them all at handling the heat of the components they were using. Until the M1 Pro & Max rolled around, anyway, and suddenly they got thicker, with feet that raise it farther off the desk, and absolutely massive amount of vents all over 3 sides of the machine. Curious timing on that...

And of course, Apple has made huge progress since then (M1, better thermal designs, new fan designs which are quieter and more efficient) whereas PC makers have made basically zero progress.

The "huge progress since then" was a year ago.

Your timeline is a little bit made up.

https://www.theverge.com/22967776/apple-magic-mouse-charging...

Once again we get a Mac Semi-Pro Mini (seems like the Studio is more like a replacement for the Trashcan) that their marketing implies is maybe as good as a Mac Pro but is obviously not. It does look a lot better this time around - at least it has more ports :-D

Thank you for recalibrating me to actual reality and not Apple Reality (tm)

https://www.techpowerup.com/64683/nvidia-admits-to-selling-f...

Apple specifically: https://support.apple.com/en-us/HT203254

Anyway, it was still my longest lived Laptop. My Sony VAIOs were great but I liked that Mac better.

Can we stop it with the meme that these GPUs are unobtainable? Yes, they are still overpriced compared to their supposed original prices and they'll likely never return to that price given that the base prices of manufacturing, materials and such have increased for multiple reasons.

But stock has been generally available for many months now and it's possible to get them as long as you can afford them.

Don't know if it's the same, these days, but when I was designing electronic stuff, we were always told to spec the power supply at twice the maximum draw.

And that is expected, a lot has to be reserved for USB devices.

No, not really. You can try adding sound dampening, like BeQuiet does, but it's not as effective as just having more lower RPM fans (although it does help with coil whine). But Apple has historically never used sound dampening, and this doesn't look like it changes that. With how "open" it is anyway (the entire back being just a bunch of holes), sound dampening wouldn't be all that effective.

I'm not really sure what you think the heavier heat sink has to do with either the fan RPM or the noise profile. The bigger heatsink is if anything evidence of larger, lower-RPM fans. They're using more surface area, so they can spread the air movement out over larger fan blades. Which means they don't need to use as high an RPM fan.

There is raw performance, but there is also performance per watt, availability and scalability (which is both good and bad - M1 is available, but there is no M1 Ultra cloud available). If you want a multi-use setup, an RTX makes more sense than most other options, if you can get one and at a reasonable price. If you want a Mac, the M1U is going to give you the best GPU. In pretty much all other setups there are so many other variables it's hard to recommend anything.

If you are doing it solo, with "just the hardware you happen to have", it matters a bit less. If you are doing it constantly to make money, and you need a lot of power, buying a one-person desk-machine makes no sense.

Even assuming literal 24hr/day usage at a higher, "factory overclocked" 450w sustained, at a fairly high $0.30/kWh that's $1200/yr. Less than half the retail price of a 3090. And you can easily drop the power limit slider on a 3090 to take it down to 300-350w, likely without significantly impacting your workload performance. Not to mention in most countries the power cost is much less than $0.30/kWh.

At a more "realistic" 8 hours a day with local power pricing I'd have to run my 3090 for nearly 10 years just to reach the $2000 upgrade price an M1 Ultra costs over the base model M1 Max.

At $0.36/kWh - card alone @450W ~running cost ~= RRP 3090 $1,499

Yes, can power it down to be more efficient, however, that effectively agrees with the previous comment that PPW matters.

Watts are dollars that you'll continue spending over the system life. It matters because you can only draw so many amps per rack and there will be a point when, in order to get more capacity, you'll need to build another datacenter.

These machines are commonly used by professionals in industries like movie and music production. They don't care what the power bill is, it's insignificant compared to the cost of the employee using the hardware.

Oh... They do. At least, they should. If a similar PC costs $500 less but you spend $700 more in electricity per year because of it, at the end of the year, your profits will show the difference.

I ran a visual effects company a decade ago. We bought the fastest machines we could because saving time for production was important. The power draw was never a factor; a few catered lunches alone would dwarf the power bill.

The silly thing about it is that most of the special engines can now be flashed into an FPGA which is becoming more common in the big clouds so special offload engines aren't that big of a deal when they are missing. So in some cases you can have your cake and eat it too; massive parallel processing and specialised processing in the same server box without resorting to special tricks (as long as it's not suddenly getting blocked in future software updates).

Honestly asking because I’m kind of out of the nvidia loop at the moment.

Legally: I assume no cloud provider will assume the legal risk of telling their customers "and here you have to break the EULA of the NVIDIA driver in that way to use the service". In Europe where the legal environment is more focused on interoperability, this might not be as much of a problem, but still it may be too much risk.

Even though it's not an apples to apples comparison, keep in mind that a 1x32GB DIMM sells for less than 150$, and you can buy 1TB SSDs for less than 100$.

> keep in mind that a 1x32GB DIMM sells for less than 150$

Keep in mind that M1 Pro/Max/Ultra is LPDDR5 6400 (https://www.anandtech.com/show/17024/apple-m1-max-performanc...) connected by a 512 bit memory controller.

Whereas is a kit of 2x 32 GB LPDDR5 4800 (I could not easily locate a quote for 1x 64Gb LPDDR5 4800 DIMM, leave alone 6400) retails for USD 548 (https://www.newegg.com/crucial-64gb-288-pin-ddr5-sdram/p/N82...).

I could not locate a reliable source on the type of the SSD employed in M1 Pro/Max/Ultra, so I will refrain from remarking on the comparison.

And what WD advertises for the Black SN850: https://www.westerndigital.com/products/internal-drives/wd-b...

And what Seagate advertises for the FireCuda 530: https://www.seagate.com/products/gaming-drives/pc-gaming/fir...

And what Gigabyte advertises for the Gen4 7000s: https://www.gigabyte.com/Solid-State-Drive/AORUS-Gen4-7000s-...

etc...

They aren't $100 for 1TB, no, but a lot of them are around $150. Which would be a lot less than +$100 to go from 512GB to 1TB, too. It's $40 to go from the 500GB SN850 to the 1TB SN850, for example.

When configured to ensure data integrity in the case of power loss (more important in this new M1 Studio machine unless it comes with integrated battery), then it's a lot worse.

edit: just found out - it's NBB:

https://www.notebooksbilliger.de/

There's probably a German discord somewhere to have alert for drops.

They're overpriced for sure, and that's the only reason the M1U pricing looks equivalent rather than exorbitant

I wasn’t a DVer, but they’ve always been available if you were willing to pay a scalper. The only thing that has changed is that more retailers find it appropriate to rip off their customers. It’s kinda like a liquor store I’ve done business with for years now wanting $3,800 for a bottle of 23yr PVW. MSRP is $299.99. Should the owner not be able to make extra profit on it, no, of course they should. But >12x MSRP is just predatory imho.

* Much higher than MSRP, which was reduced compared to the previous generation because that attempt at raising prices killed market demand for said previous generation.

* No longer affordable by the traditional customer base but sustained by a new market with questionable longevity in its demand

Take your pick.

Sure, in a pure rational economic sense the market price has risen because supply has fallen at the same time a new market of buyers became very interested in the product, but we're talking consumer expectations and historical trends here, not the current price in a vacuum.

Interested to know what you think a reasonable PSU would be for A machine that was consuming close to 200W for processing...