Since you mention ML specifically, looking at some benchmarks out there (like https://tlkh.dev/benchmarking-the-apple-m1-max#heading-gpu & https://wandb.ai/tcapelle/apple_m1_pro/reports/Deep-Learning... ), even if the M1 Ultra is 2x the performance of the M1 Max (so perfect scaling), it would still be far behind the 3090. Like completely different ballpark behind. But of course there is that price & power gap, but the primary strength of the M1 GPUs seems to really be from the essentially very large VRAM amount. So if your working set doesn't fit in an RTX GPU of your desired budget, then the M1 is a good option. If, however, you're not VRAM limited, then Nvidia still offers far more performance.

Well, assuming you can actually buy any of these, anyway. The M1 Ultra might win "by default" by simply being purchasable at all unlike pretty much every other GPU :/

Back when that M1 MAX vs 3090 blog post was released, I ran those same tests on the M1 Pro (16GB), Google Colab Pro, and free GPUs (RTX4000, RTX5000) on the Paperspace Pro plan.

To make a long story short, I don't think buying any M1 chip make senses if your primary purpose is Deep Learning. If you are just learning or playing around with DL, Colab Pro and the M1 Max provide similar performance. But Colab Pro is ~$10/month, and upgrading any laptop to M1-Max is at least $600.

The "free" RTX5000 on Paperspace Pro (~$8 month) is much faster (especially with fp16 and XLA) than M1 Max and Colab Pro, albeit the RTX5000 isn't always available. The free RTX4000 is also a faster than M1 Max, albeit you need to use smaller batch sizes due to 8GB of VRAM.

If you assume that M1-Ultra doubles the performance of M1-Max in similar fashion to how the M1-Max seems to double the gpu performance of the M1-Pro, it still doesn't make sense from a cost perspective. If you are a serious DL practitioner, putting that money towards cloud resources or a 3090 makes a lot more sense than buying the M1-Ultra.

Apple Silicon (including base M1) actually has great FP16 support at the hardware level, including conversions. So it is wrong to say it only supports FP32.

> The Core ML runtime dynamically partitions the network graph into sections for the Apple Neural Engine (ANE), GPU, and CPU, and each unit executes its section of the network using its native type to maximize its performance and the model’s overall performance. The GPU and ANE use float 16 precision, and the CPU uses float 32.

Also, this exploration (https://tlkh.dev/benchmarking-the-apple-m1-max#heading-neura...) reports the 5.1-5.3 TFLOPS FP16 ballpark performance.

No one plays games on a Mac.

And it has nothing to do with GPU performance but rather the fact that the audience simply isn't interested in gaming on it and so there is no economic incentive to target them.

So the GPU performance that matters to Mac users and is relevant to Apple is not games but rather content creation, production etc.

My wife does (I play games on Linux).

I have friends who own Macs who reluctantly dual boot to Windows just to play some games -- they would completely ditch Windows if they could just play every game on Mac.

I see there are Mac games on Steam.

All of this points to the situation being more nuanced than "no one plays games on Mac".

According to the steam hardware survey, Windows is 95%, Macos is 4%, and Linux is 1%. And to dig deeper you'd need to see what games that 5% of the non-Windows is playing - are they simpler games that don't need graphics acceleration (e.g. puzzle games, roguelikes, etc) or ones that do?

My desktop is an Intel NUC running Ubuntu, and yeah I play games on it. Slay the Spire, Spacechem, even some older MMOs like DDO or LoTRO (which run but at 15-20 fps since that system just has Intel Iris). I'm unable to even start many others (e.g. Grim Dawn) due to not having dedicated graphics.

So yeah it's nuanced but lots of games that need a graphics card don't run or even display on that system.

That's why I have a windows gaming system too. I'm realistic, the market just isn't there. I used to have a Mac (dropped it in 2018) and if I still did I'd subscribe to Geforce Now or just do console gaming.

Who says Mac users aren't gamers? It's a self perpetuating vicious cycle: gamers use Windows because that's where the majority of games are, so developers keep targeting Windows, so gamers use Windows, and so ad infinitum.

But people who use Macs enjoy games as well. They would rather not dual boot to Windows.

It's not true that Mac users don't play games. Rather, it's that most games are on Windows, which is a shame.

What I think you mean is competitive gamers and AAA gamers do not play on Apple hardware. This is mostly true today ofc, but keep in mind that's actually not the majority of the gaming market. Apple is raking it in from its gaming market.

I have no idea what the answer is. Personally I have run a number of games on macOS, via Steam, and via Boot Camp and virtualization. Some popular MMOs like Final Fantasy XIV, World of Warcraft, and Eve Online have macOS clients, though Guild Wars 2 discontinued theirs.

Apple Arcade apparently has enough Mac users that Apple has a reason to support it on macOS as well as iOS.

And Apple apparently has some reason to support iOS/iPadOS games on Apple Silicon Macs as well (though it could just be a side effect of a future iOS-macOS merger or hybrid device.)

...and how many would play more games on their Macs if they were available?

I refuse to believe Mac users are less fond of videogames. Based on my personal observation of the Mac users I know, they enjoy games as much as anyone.

At least casual games on iTunes for the iPad have a vast library, with many genuine good games (source: me, my iPad 2 was mainly a gaming platform for me, never found other uses for it).

I do realize more complex games are a different beast. But casual? I'd say Apple fans love them.

Video content creation specifically is where it mostly achieves what the graphs indicate, and that's mostly a "well yeah, video decoder ASICs are really efficient. I'll take things we already knew for $100, Alex"

If the software is optimized, the graphics hold up fine.

Won't even get into that it can't run most things you'd want that kind of hardware for.

edit: Of course the M1 max only shipped in laptops, so... who knows.

I’d assume that’s what most of the chonk is about, no?

> Based on the thermal and power consumption characteristics of previous chips I would not be surprised if say ~120W is the max power draw of this thing.

The Max could be brought up to 90W or so.

If you think this is still a problem, you haven't used any recent Macs. The current MB Air and MB Pro both run very cool even under prolonged heavy loads.

Apple's management of any and all heat issues has been far better than any competitors for a while now.

Only if you define "for a while" as "since a year ago with the introduction of the M1".

Apple refused to make a thicker laptop or one with better ventilation to adequately cool the CPUs & GPUs they were sticking in them. They were among if not the worst of them all at handling the heat of the components they were using. Until the M1 Pro & Max rolled around, anyway, and suddenly they got thicker, with feet that raise it farther off the desk, and absolutely massive amount of vents all over 3 sides of the machine. Curious timing on that...

And of course, Apple has made huge progress since then (M1, better thermal designs, new fan designs which are quieter and more efficient) whereas PC makers have made basically zero progress.

The "huge progress since then" was a year ago.

Your timeline is a little bit made up.

https://www.theverge.com/22967776/apple-magic-mouse-charging...

Once again we get a Mac Semi-Pro Mini (seems like the Studio is more like a replacement for the Trashcan) that their marketing implies is maybe as good as a Mac Pro but is obviously not. It does look a lot better this time around - at least it has more ports :-D

Thank you for recalibrating me to actual reality and not Apple Reality (tm)

https://www.techpowerup.com/64683/nvidia-admits-to-selling-f...

Apple specifically: https://support.apple.com/en-us/HT203254

Anyway, it was still my longest lived Laptop. My Sony VAIOs were great but I liked that Mac better.

Can we stop it with the meme that these GPUs are unobtainable? Yes, they are still overpriced compared to their supposed original prices and they'll likely never return to that price given that the base prices of manufacturing, materials and such have increased for multiple reasons.

But stock has been generally available for many months now and it's possible to get them as long as you can afford them.

Don't know if it's the same, these days, but when I was designing electronic stuff, we were always told to spec the power supply at twice the maximum draw.

And that is expected, a lot has to be reserved for USB devices.

No, not really. You can try adding sound dampening, like BeQuiet does, but it's not as effective as just having more lower RPM fans (although it does help with coil whine). But Apple has historically never used sound dampening, and this doesn't look like it changes that. With how "open" it is anyway (the entire back being just a bunch of holes), sound dampening wouldn't be all that effective.

I'm not really sure what you think the heavier heat sink has to do with either the fan RPM or the noise profile. The bigger heatsink is if anything evidence of larger, lower-RPM fans. They're using more surface area, so they can spread the air movement out over larger fan blades. Which means they don't need to use as high an RPM fan.

There is raw performance, but there is also performance per watt, availability and scalability (which is both good and bad - M1 is available, but there is no M1 Ultra cloud available). If you want a multi-use setup, an RTX makes more sense than most other options, if you can get one and at a reasonable price. If you want a Mac, the M1U is going to give you the best GPU. In pretty much all other setups there are so many other variables it's hard to recommend anything.

If you are doing it solo, with "just the hardware you happen to have", it matters a bit less. If you are doing it constantly to make money, and you need a lot of power, buying a one-person desk-machine makes no sense.

Even assuming literal 24hr/day usage at a higher, "factory overclocked" 450w sustained, at a fairly high $0.30/kWh that's $1200/yr. Less than half the retail price of a 3090. And you can easily drop the power limit slider on a 3090 to take it down to 300-350w, likely without significantly impacting your workload performance. Not to mention in most countries the power cost is much less than $0.30/kWh.

At a more "realistic" 8 hours a day with local power pricing I'd have to run my 3090 for nearly 10 years just to reach the $2000 upgrade price an M1 Ultra costs over the base model M1 Max.

At $0.36/kWh - card alone @450W ~running cost ~= RRP 3090 $1,499

Yes, can power it down to be more efficient, however, that effectively agrees with the previous comment that PPW matters.

Watts are dollars that you'll continue spending over the system life. It matters because you can only draw so many amps per rack and there will be a point when, in order to get more capacity, you'll need to build another datacenter.

These machines are commonly used by professionals in industries like movie and music production. They don't care what the power bill is, it's insignificant compared to the cost of the employee using the hardware.

Oh... They do. At least, they should. If a similar PC costs $500 less but you spend $700 more in electricity per year because of it, at the end of the year, your profits will show the difference.

I ran a visual effects company a decade ago. We bought the fastest machines we could because saving time for production was important. The power draw was never a factor; a few catered lunches alone would dwarf the power bill.

The silly thing about it is that most of the special engines can now be flashed into an FPGA which is becoming more common in the big clouds so special offload engines aren't that big of a deal when they are missing. So in some cases you can have your cake and eat it too; massive parallel processing and specialised processing in the same server box without resorting to special tricks (as long as it's not suddenly getting blocked in future software updates).

Honestly asking because I’m kind of out of the nvidia loop at the moment.

Legally: I assume no cloud provider will assume the legal risk of telling their customers "and here you have to break the EULA of the NVIDIA driver in that way to use the service". In Europe where the legal environment is more focused on interoperability, this might not be as much of a problem, but still it may be too much risk.

Even though it's not an apples to apples comparison, keep in mind that a 1x32GB DIMM sells for less than 150$, and you can buy 1TB SSDs for less than 100$.

> keep in mind that a 1x32GB DIMM sells for less than 150$

Keep in mind that M1 Pro/Max/Ultra is LPDDR5 6400 (https://www.anandtech.com/show/17024/apple-m1-max-performanc...) connected by a 512 bit memory controller.

Whereas is a kit of 2x 32 GB LPDDR5 4800 (I could not easily locate a quote for 1x 64Gb LPDDR5 4800 DIMM, leave alone 6400) retails for USD 548 (https://www.newegg.com/crucial-64gb-288-pin-ddr5-sdram/p/N82...).

I could not locate a reliable source on the type of the SSD employed in M1 Pro/Max/Ultra, so I will refrain from remarking on the comparison.

And what WD advertises for the Black SN850: https://www.westerndigital.com/products/internal-drives/wd-b...

And what Seagate advertises for the FireCuda 530: https://www.seagate.com/products/gaming-drives/pc-gaming/fir...

And what Gigabyte advertises for the Gen4 7000s: https://www.gigabyte.com/Solid-State-Drive/AORUS-Gen4-7000s-...

etc...

They aren't $100 for 1TB, no, but a lot of them are around $150. Which would be a lot less than +$100 to go from 512GB to 1TB, too. It's $40 to go from the 500GB SN850 to the 1TB SN850, for example.

When configured to ensure data integrity in the case of power loss (more important in this new M1 Studio machine unless it comes with integrated battery), then it's a lot worse.

edit: just found out - it's NBB:

https://www.notebooksbilliger.de/

There's probably a German discord somewhere to have alert for drops.

They're overpriced for sure, and that's the only reason the M1U pricing looks equivalent rather than exorbitant

I wasn’t a DVer, but they’ve always been available if you were willing to pay a scalper. The only thing that has changed is that more retailers find it appropriate to rip off their customers. It’s kinda like a liquor store I’ve done business with for years now wanting $3,800 for a bottle of 23yr PVW. MSRP is $299.99. Should the owner not be able to make extra profit on it, no, of course they should. But >12x MSRP is just predatory imho.

* Much higher than MSRP, which was reduced compared to the previous generation because that attempt at raising prices killed market demand for said previous generation.

* No longer affordable by the traditional customer base but sustained by a new market with questionable longevity in its demand

Take your pick.

Sure, in a pure rational economic sense the market price has risen because supply has fallen at the same time a new market of buyers became very interested in the product, but we're talking consumer expectations and historical trends here, not the current price in a vacuum.

Interested to know what you think a reasonable PSU would be for A machine that was consuming close to 200W for processing...

1. https://techguided.com/best-rtx-3090-gaming-pc/#:~:text=With....

That 3500 is for a DIY build. So, sure, you can always save on labor and hassle, but prebuilt 3090 rigs commonly cost over 4k. And if you don't want to buy from Amazon because of their notorious history of mixing components from different suppliers and reselling used returns, oof, good luck even getting one.

So eventually you still have to "replace everything" to upgrade a PC.

You could also put a 4790k, 16gb of ddr3 and a modern gpu in that system to get a perfectly functional gaming system that will do most titles on 1080p high. Though admittedly we've passed the point where that's financially sensible vs upgrading to a 12400 or something as both devil's canyon CPUs and ddr3 are climbing back up in price as supplies diminish

DDR4 was released in 2014, which would suggest you purchased your mobo two full years after DDR3 was already deemed legacy technology and being phased out.

Also LGA1150 was succeeded by LGA1151 in 2015, which means you bought your mobo one full year after it was already legacy hardware.

Point being, PC-building makes it easier to replace and repair individual components, but in time, upgrading to newer generations means spending over 50% of the original cost on motherboard, CPU, PSU, RAM. Not too different than dropping $3K on a new Mac.

[1] https://web.archive.org/web/20101219085440/http://www.xbitla...

It means the hardware was purchased after it started to be discontinued.

It's hardly a reasonable take, and makes little sense, to complain how you can't upgrade hardware that was already being discontinued before you bought it.

> DDR3 and LGA1150 were not deemed "legacy" the day DDR4 and LGA1151 motherboards entered the market.

I googled for LGA1150 before I posted the message, and one of the first search results is a post on Linux tech tips dating way back to 2015 on whether LGA1150 was already dead.

And you purchased the Mobo one year after that.

Also, we must have a different interpretation of "discontinued", because DDR3 and LGA1150 were still produced, sold, and dominated sales for way long after I bought that system. At the time (and for the next 1-2 years), consumer DDR4 was a luxury component that most no existing hardware supported.

To do CPU upgrades you eventually have to replace the motherboard but you can keep using whatever your GPU/storage/other parts is. Sometimes that also means a RAM upgrade but it's still better than the literal nothing of modern Macs.

EDIT: I’m quite confident this is not at all an exaggeration. Unless you have put together PCs for a living. $100/h (total employment cost, not just salary), 1-2 hours of actual build & setup, 8 more hours of speccing out parts, buying, taking delivery, installing stuff and messing around with windows/Linux (I’ve probably spent 40 hours+ in the past couple years just fixing stuff in my windows gaming pc. At least 1 of those looking for a cabled keyboard so I could boot it up the first time, ended up having a friend drive over with his :D)

You have to spec out the parts, ensuring compatibility. Manage multiple orders and deliveries. Assemble it. Install drivers/configuration specific packages.

All of these things are easier today than ten or twenty years ago - but assigning it to a random mid-level engineer and I'd set my project management gamble on half a day for the busiest, most focused engineers least likely to take the time to fuss over specs, or one day for the majority.

ofc. to get to $1000 for that they'd still have to be on $230k to $460k.

In my case I have built a couple PCs before, but it was so long ago that I'd have to re-learn which retailers are trustworthy, what the new connection standards are these days, etc. It's just not worth it to me to spend a dozen hours learning, specing, ordering, assembling, installing, configuring, etc to save a few hundred bucks.

A senior engineer in the Bay can easily pull down $400k/year in total comp, which is $200/hour. The rule of thumb I've always heard is that a fully-loaded engineer costs roughly 2x their comp in taxes/insurance/facilities/etc.

When someone costs the company north of $3k/day, it's cheaper all round to just plonk a brand new $6k MacBook Pro on their desk if they have a hardware issue.

(I've been accused of overuse of acronyms, but that one's rare!)

Performance per watt? I could see that being disrupted, but an iGPU in 2022 will be orders of magnitude less powerful than a dGPU, if wattage is ignored.

If Nvidia were on the same node and increased die space to match M1 (ignoring the CPU portion of the die size), they would then be able to run at a lower clock with more compute units and probably match the TDP discrepancy.

An iGPU isn't necessarily slower if the system ram is fast, and M1 was one of the first consumer CPUs to move to DDR5. 3090 has 936.2 GB/s with GDDR6X, M1 Ultra with DDR5 memory controllers on both dies gets 800GB/s.

For the record: I was the first M1 recipient (temporary 16gb MB, stock issues). I needed an Intel MBP because Rosetta ain't all that. I opted for, and was upgraded to, the 32gb M1 MBP. I chose M1 over Intel because it was unbelievably faster for the form-factor. My original comment does not concern laptops. My PC is orders of magnitude more powerful.

TDP is physics. You all might perceive Apple as perfect and infallible and all so lovely, but physics is physics.

I use AMD, not NVIDIA. And "what if" is irrelevant. It's like intentionally neutering Zen2 by comparing it to Intel single-core (as was done all the time). The reality is absolute, not relative. Comparing effective performance, not per-TDP, is what matters to the user. And my network/gpu/audio drops on both my 16gb M1 MB and 32gb M1 MBP under load.

Seriously not buying that "Apple can do nothing wrong" bias.

Take those #Ff6600-colored glasses off. The M1 has unbeatable value proposition in a pretty wide market, but Apple couldn't be further from a universally good machine.

Here in Australia, 3090’s go for close to 3k on their own.

20-Core CPU 48-Core GPU 32-Core Neural Engine

    64GB unified memory
    1TB SSD storage¹
    Front: Two Thunderbolt 4 ports, one SDXC card slot
    Back: Four Thunderbolt 4 ports, two USB-A ports, one HDMI port, one 10Gb Ethernet port, one 3.5-mm headphone jack

But even ignoring the FE series, the prices have already crashed massively, you can get a 3080 AIB for less than £1000, and 3090s frequently appear around £1500-1600.

But you’re comparing apples to oranges, because the real advantage of M1 chips is the unified memory - almost no CPU-GPU communication overhead, and that the GPU can use ginormous amounts of memory.

The GPU claims wouldnt even need to be on parity with NVIDIA, it would just need to offer a vertically integrated alternative to having to use EC2.

What reliability issues are you having with TensorFlow on M1 Macs?

Now i've got a team of data scientists in a fully MBP shop and we're holding off upgrades to M1 until this all gets resolved.

On my personal M1, I managed to make it work, but its hard to know the layers of changes made and what exactly allowed it to work.

You can buy single tensor accelerators from Google: https://www.coral.ai/products/

You can buy a bunch of those integrated into a single PCI-E card. https://iot.asus.com/products/AI-accelerator/AI-Accelerator-...

Cheap too. Some of these work with Mac. More of them work for PC, because the hardware interface is outside of Apple's thin vertical slice/garden.

I really hope I'm wrong (as someone who owns an M1 Pro chip) but I find it hard to imagine things changing significantly in the next ~2 years unless someone is able (legally and technically) to release a CUDA compatibility layer.

Naturally, the HIP tooling doesn't support M1 GPUs at this time. We'll see if anyone else tries.

"M1 Ultra has a 64-core GPU, delivering faster performance than the highest-end PC GPU available, while using 200 fewer watts of power."

Note that they say "faster performance" not "more performance". What does "faster" mean? Who knows!

I still think “faster performance” sounds sound odd, but I understand their point.

5-10 years ago they were still serious about open standards, like OpenCL.. Now it's all locked in.

My guess is they're putting a lot of weight on the phrase "relative power" and hoping you assume it means "relative to each other" and not "relative to their previous generation" (i.e. M1 Ultra -> M1 Max and RTX 3090 -> RTX 2080Ti) or "relative to the stock power profile".

Put bluntly, if the M1 Ultra was capable of achieving performance parity with an RTX 3090 for any GPU-style benchmark then Nvidia (who are experts in making GPUs) would have captured this additional performance. Bear in mind the claim seems to be (on the surface) that the M1 Ultra is achieving with 64 GPU cores and 800GB/s memory bandwidth what the RTX 3090 is achieving with 10,496 GPU cores and 936.2GB/s memory bandwidth.

But you'll all but certainly see the 3090 win more benchmarks (and by a landslide) than the M1 Ultra does. Because Nvidia is really, really fucking good at this, and they spend an absurd amount of money working with external projects to fix their stuff. Like contributing a TensorFlow backend for CUDA. Or tons of optimizations in the driver to handle game-specific issues.

Meanwhile Apple is mostly in the camp of "well we built Metal2, what's taking ya'll so long to port it to our tiny marketshare platform that historically had terrible GPU drivers?"

It is also worth noting that the M1 Ultra is an SoC so it'll have more than just CPU/GPU on it, by the looks of things it has some hefty amounts of cache, it'll also have a few IP blocks like a PCIe controller, memory controller, SSD controller (the current "SSDs" look to just be raw storage modules).

All told it likely still has somewhere in the region of 30-40 billion transistors for the GPU. Each GPU core being physically bigger than the 3090 is probably pretty good for some workflows and not so good for others. Generally GPUs benefit from having a huge number of tiny cores for processing in parallel, rather than a small number of massive cores.

Current benchmarks put it at roughly the performance of an RTX 3070, which is good for its power consumption, but not even close to the 3090. As I mentioned in the previous post, it just doesn't have the cores or memory bandwidth needed for the types of workloads that GPUs are built for (although unified memory being physically closer can help here ofc.), certainly not enough to make it a competitor for something like a 3090.

Edit: Oh also, for massively parallel workloads (like what GPUs do), more cores and better bandwidth to feed those cores will be one of the biggest performance drivers. You can get more performance by making those cores bigger (and therefore faster) but you need to crank the transistor count up a _lot_ to match the kinds of throughput that many tiny cores can do.