I don't believe that's true, regardless of what you mean by "can run at a power envelope".

The M1/M2/M3 use 20-25 Watt at peak, but only 10% of that when idle. (But that's only in the notebooks – it goes beyond that in the Mac mini.)

The M3 Pro is basically 50% more of everything than the M3, so I'd reuse that multiplier to guess its power levels.

Parent (and the article) are talking about power per core.

If the rumors about LLM stuff baked into Siri on the next iPhone are true, it will be interesting see how they make the most use of the hardware with that.

I'm really hoping for a complete replacement. Talk about a product that went downhill over time.

Literally changes throughout the day from sort of working to insulting me by doing the opposite ("turn on the kitchen lights" -> all lights in the house turn off)

What's interesting is the A17 Neural Engine is about twice as powerful as it in on the M3 series - 35 TOPS vs. 18.

It's rumored the difference is to aid in photography and possibly Face ID, but of course this will be helpful for any onboard inference.

even if that's true, the A series neural engine is much worse than the M series, I'll say it wrong, but it can only do 32bit inference (or something like that) where M series can do 64bit, so A series can run LLM but has a series of limitations that M series doesn't.

Practically speaking, most models today infer at 8bit or 16bit (sometimes, rarely 32). You don't see an empirical lift at more bits of precision. Size of the memory is far more important.

If we're talking about the results, is there any reason to think it should make a difference at all?

Sometimes gradients are small but meaningful, if you constrain them to too few bits / degrees of freedom they'll be unable to backprop successfully. This can hamper training and therefore results quality.

You can also think about it as compounding errors - at any one weight index the bit values might not be too meaningful, but cascaded over a lot of tensor multiplications they will be.

Oh I was thinking we were talking about the same calculations on different hardware.

I think next year's refreshes across the board will have a different architecture so the neural engine has a lot more access to RAM so that models with larger parameters can be used and faster

Good Lord this may the brain transplant she has needed for so long.

I thought you were piling on a human and felt a bit outraged.

Siri deserves more slander than you delivered.

Too slow to be useful for anything interesting.

Too expensive to be useful for any IOT or general computing.

I'm sure its really useful that one time you couldn't take a flight to a location and didn't have a car so you took a bus and that bus was made in 1990 and didn't have 120V outlets retrofitted and you decided to work for 12 hours straight on things that never used GPU.

Haha true that. It's really limited in terms of how well it can be integrated, however I really wish Apple would make some socket version of this(in future )for general consumers or atleast OEMs , it would be hugely successful.

The chances are close to impossible since it's Apple, and the architecture will need lot's of changes or the OEMs will have to change their motherboard design drastically to support a socketed M series chip ( storage controller, memory controller are all on the SOC)

Predictively you got massively downvoted but I don't understand how that is not the sentiment. People are really irrational around Apple stuff. Funniest thing is that I like macOS enough that I would buy their crap if it wasn't so absurdly expansive because of RAM pricing. I'm willing to let go of gaming and GPU apps (wasn't many in macOS anyway, because there were historically behind) but come on, not at this price... The battery life argument is a mystery to me. I wish they would figure out this would much better in a product like the Watch, instead of wasting power for useless computation, graphics or what have you...

Without taking away from how cool that is, the Threadripper 7980X manages to run at just 3.4W/core according to GamersNexus [1]

It's pretty amazing where everyone (other than Intel) is on the perf/W scale.

[1] https://www.youtube.com/watch?v=yDEUOoWTzGw

I don’t know but I’m curious: I’d imagine at that many cores you’re probably reaching the limits of how much power the socket can realistically get? I know some motherboards have that supplemental power port.

Yeah motherboards for threadripler have extra power supply connections.

I believe there's a the option to connect two

https://videocardz.com/newz/amd-threadripper-pro-7995wx-96-c...

World record on cinebench with liquid nitrogen pulled 1.6kW

As I understand it, the main issue is cooling the damn thing. Socket SP5 can do up to 700W, and that's all emitted from a heatspreader 72mm x 75mm.

Getting the power into it isn't trivial, but it's not rocket science either. When the socket has 6000 pins, you can spare a few hundred for power.