News for hackers. Stuff that matters.
Re:Benchmark Framework Desktop Mainboard and 4-node cluster by geerlingguy on Thursday August 07, 2025 @09:55PM (#44832600)
108 GB or so under Linux.

The BIOS allows pre-allocating 96 GB max, and I'm not sure if that's the maximum for Windows, but under Linux, you can use `amdttm.pages_limit` and `amdttm.page_pool_size` [1]

[1] https://www.jeffgeerling.com/blog/2025/increasing-vram-alloc...

Re:Benchmark Framework Desktop Mainboard and 4-node cluster by amstan on Friday August 08, 2025 @02:02AM (#44833944)
I have been doing a couple of tests with pytorch allocations, it let me go as high as 120GB [1] (assuming the allocations were small enough) without crashing. The main limitation was mostly remaining system memory:

    htpc@htpc:~% free -h
                   total        used        free      shared  buff/cache   available
    Mem:           125Gi       123Gi       920Mi        66Mi       1.6Gi       1.4Gi
    Swap:           19Gi       4.0Ki        19Gi
[1] https://bpa.st/LZZQ
Re:Benchmark Framework Desktop Mainboard and 4-node cluster by cpburns2009 on Friday August 08, 2025 @09:31AM (#44836719)
Thanks for the correction. I was under the impression the GPU memory had to be preallocated in the BIOS, and 96 GB was the maximum number I read about.
Re:Benchmark Framework Desktop Mainboard and 4-node cluster by sliken on Saturday August 09, 2025 @12:48AM (#44844100)
Some older software stacks require static allocation in BIOS, but things are moving pretty quickly and allow dynamic allocation. Newer versions (or patches to) pytorch, ollama, and related, which I think might depend on a newer kernel (6.13 or so). Does seem like there's been quite a bit of progress in the last month.