News for hackers. Stuff that matters.
Re:Benchmark Framework Desktop Mainboard and 4-node cluster by cpburns2009 on Thursday August 07, 2025 @08:34PM (#44832106)
Just a point of clarification. I believe the 128GB Strix Halo can only allocate up to 96GB of RAM to the GPU.
Re:Benchmark Framework Desktop Mainboard and 4-node cluster by geerlingguy on Thursday August 07, 2025 @09:55PM (#44832600)
108 GB or so under Linux.

The BIOS allows pre-allocating 96 GB max, and I'm not sure if that's the maximum for Windows, but under Linux, you can use `amdttm.pages_limit` and `amdttm.page_pool_size` [1]

[1] https://www.jeffgeerling.com/blog/2025/increasing-vram-alloc...

Re:Benchmark Framework Desktop Mainboard and 4-node cluster by amstan on Friday August 08, 2025 @02:02AM (#44833944)
I have been doing a couple of tests with pytorch allocations, it let me go as high as 120GB [1] (assuming the allocations were small enough) without crashing. The main limitation was mostly remaining system memory:

    htpc@htpc:~% free -h
                   total        used        free      shared  buff/cache   available
    Mem:           125Gi       123Gi       920Mi        66Mi       1.6Gi       1.4Gi
    Swap:           19Gi       4.0Ki        19Gi
[1] https://bpa.st/LZZQ
Re:Benchmark Framework Desktop Mainboard and 4-node cluster by cpburns2009 on Friday August 08, 2025 @09:31AM (#44836719)
Thanks for the correction. I was under the impression the GPU memory had to be preallocated in the BIOS, and 96 GB was the maximum number I read about.
Re:Benchmark Framework Desktop Mainboard and 4-node cluster by sliken on Saturday August 09, 2025 @12:48AM (#44844100)
Some older software stacks require static allocation in BIOS, but things are moving pretty quickly and allow dynamic allocation. Newer versions (or patches to) pytorch, ollama, and related, which I think might depend on a newer kernel (6.13 or so). Does seem like there's been quite a bit of progress in the last month.
Re:Benchmark Framework Desktop Mainboard and 4-node cluster by lhl on Friday August 08, 2025 @03:37AM (#44834539)
In Linux, you can allocate as much as you want with `ttm`:

In 4K pages for example:

    options ttm pages_limit=31457280
    options ttm page_pool_size=15728640
This will allow up to 120GB to be allocated and pre-allocate 60GB (you could preallocate none or all depending on your needs and fragmentation size. I believe `amdgpu.vm_fragment_size=9` (2MiB) is optimal.