News for hackers. Stuff that matters.

A comment on Run a 1T parameter model on a 32gb Mac by streaming tensors from NVMe. Parent comment

Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by zozbot234 on Tuesday March 24, 2026 @01:26PM (#47506188)
> The kernel's page fault handler is reactive — it doesn't know you're about to read layer 47's FFN weights, so it can't prefetch.

man 2 madvise

[ Parent | on HN ]
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by astrange on Tuesday March 24, 2026 @05:40PM (#47509801)
That works for readahead but it's not good for random access. readv, aio, dispatch_io are better there.
[ Parent | on HN ]
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by zozbot234 on Tuesday March 24, 2026 @05:54PM (#47509983)
This claim is a bit apples and oranges (no pun intended!). madvise is all about providing hints to the kernel to tune the page cache and readahead (including possibly disabling readahead altogether). it's not about performing reads into private memory buffers, which is actually where the options you mentioned fit in.
[ Parent | on HN ]
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by astrange on Wednesday March 25, 2026 @11:08AM (#47518391)
Triggering reads is also how you get pages into the page cache, so it helps to know how to do it.
[ Parent | on HN ]