The M5 SSD's performance uplift was fairly substantial, even when compared to the prior generation.

> In the Blackmagic Disk Speed Test, the SSD in the M5 MacBook Pro achieved read speeds of up to 6,323 MB/s, compared to just 2,031 MB/s on the M4 MacBook Pro. It's not like the M4 is "slow" in a vacuum, but the M5 SSD is over three times faster, which is a great generation uplift.

https://www.tomshardware.com/laptops/macbooks/m5-macbook-pro...

Extremely impressive.

Not sure Data (Star Trek TNG) could read that fast.

“Accessing”

My suspicion is that this is simply due to the M5 having more memory, and the OS already having most of the file cached. The M2 has more memory pressure and would cache fewer of the SSD reads

If that's true, inference speed would be even lower if you have only 2GB total, including OS caches

The base level M5 doesn't just have more memory than the base level M2.

The memory bandwidth is bumped up by 50%, and the size of the on-die system level cache is bumped up by 50% as well.

I recently got an M5 Air as a second machine. I question if I'll even need a Pro machine in the future, as long as I can get enough RAM with the base chip (which seems unlikely, but...)

The M5 MBP has 24GB of RAM, more context in RAM perhaps?

The process stays at around 2 GB with 16 slots and a 4K context on both the M5 and M2. But yeah, Apple might be doing some magic under the hood

Unused RAM is wasted RAM. So not really Apple magic, about every OS uses "free" memory as disk cache.

Try to leave only a gigabyte or two free, speed likely would drop dramatically.

Edit: or do some calculation / logging of experts read speed, to see if it's faster than SSD spec.

yeah, looks like a page cache matters a lot I tested on mine m5 pro with 8gb memory pressure, got 27t/s instead of 35t/s Someone tested on m4 max. In regular state it was 48tok/s, but 32-42 with memory pressure

Not only is it older, so Pro:Pro it would have much slower SSD, but doesn't the Air also have slower SSD than the Pro in the same generation? And maybe narrower memory bandwidth?

IIRC, depends on the SSD size.. larger sizes had 2x the bandwidth, so it depends. That's combined/offset with the M5 improvements even further.

I am relying quite heavily on system caching and pread. And yeah, M5 is a way faster and I can guess Mac can cache something, even if process stays under 2gb.

It was 83ms read per token for M2 and 12ms on M5 pro. Total is 163ms/tok vs 30ms/tok for M5. So yeah, there is a faster read and faster gpu processing