I've got a Windows laptop with an RTX 3080 in it that runs this model no problem. I don't have it to hand or else I'd post some timings.

On my Desktop PC with a 4090 in I was getting speeds of 0.2 to 0.3 seconds for reasonably acceptable quality settings so I would expect 0.5s or so on the laptop.

What Apple are ahead on is doing this on a fanless laptop that doesn't hit internal temperatures of triple digits.

> What Apple are ahead on is doing this on a fanless laptop that doesn't hit internal temperatures of triple digits.

You also forgot the bit where Apple are ahead of doing it on a laptop that can achieve it without needing to be tethered to a power socket to achieve the performance.

It's the same thing, power is heat with talking about chips

> You also forgot the bit where Apple are ahead of doing it on a laptop that can achieve it without needing to be tethered to a power socket to achieve the performance.

Kind of sad that a huge anti-competitive, trillion dollar company is the one offering it. Especially given their stances around user freedom.

I'd much rather innovation be distributed. The goal posts should be moved to a point everyone is pushing towards the next thing. Having Apple be the only game in town is unhealthy.

Would say that rather than one company being the only one who can do it, there is only one company that can't do it, and it's Intel.

Ouch. But true.

> What Apple are ahead on is doing this on a fanless laptop that doesn't hit internal temperatures of triple digits.

I think you could pull this off on a Asus G14 in an ultra power saver mode, with the fans off or running inaudibly. The cooling is so beefy they will actually work fanless if you throttle everything down and mostly keep the GPU asleep.

The M chips could certainly sustain image generation better without a fan.

45it/s (0.1~s per image) on 7900XTX here, so it's still one magnitude faster on GPU with a lot higher power draw than the macs. Doing 10x slower with non-tethered is quite nice outcome.

At this point what Apple is ahead with is hype that M Macs are that fast, and the developers targeting them because things just work. Plenty of people should be able to run these models locally but there's close to no nice software that does that out of the box for Windows or Linux

It's because of the unified memory architecture. It's harder/different to do this on x86, because you have to have a large memory GPU and target that.

Not sure why you think it's limited to M series Macs or has to do anything with Apple at all. It's just an instruction on how to run a diffusion model trained in a novel way on particular hardware.

It's possible to do on non-Apple Silicon Macs, just more annoying. There are a few generative AI implementations which use raw Metal but not sure what the most popular one is.

It's certainly not limited to M-series Macs. Pretty much any laptop (even Macbooks) made after 2014 has AVX support for CPU acceleration. Apple's acceleration path here is pretty much a Metal shader running in Pytorch, if memory serves.

It mostly shows how shitty compatibility is between platforms that share the same roots.

The implementation is not even optimized for Macs. LCM is just very easy to be fast (batch size = 1 and only 2 to 8 steps, depending on what kind of headline you are trying to make).

They also have a decent advantage for LLMs because of their memory bandwidth to system memory vs GPU's with limited VRAM limited over PCIE to the system memory.

Got this working on an intel Mac

How?

Nevermind, I figured it out ;)