News for hackers. Stuff that matters.
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by lostmsu on Tuesday March 24, 2026 @12:47PM (#47505543)
Why would llama with --mmap crash?
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by zozbot234 on Tuesday March 24, 2026 @12:58PM (#47505746)
This doesn't surprise me all that much, mmap support gets little attention in general and interacts poorly with GPU-side inference. (And that's with it being default, you don't even really need to specify it as a CLI option.) OP has raised a discussion with the llama.cpp folks https://github.com/ggml-org/llama.cpp/discussions/20852 but little interest so far
Re:Run a 1T parameter model on a 32gb Mac by streaming tenso... by lostmsu on Wednesday March 25, 2026 @07:54AM (#47516132)
But if mmap already works why would there be any interest?

Besides, discussions are for users. He didn't open PRs or issues.