Re:Benchmark Framework Desktop Mainboard and 4-node cluster
by cpburns2009 on Thursday August 07, 2025 @08:34PM
(#44832106)
Just a point of clarification. I believe the 128GB Strix Halo can only allocate up to 96GB of RAM to the GPU.
[ Parent
| on HN ]
Re:Benchmark Framework Desktop Mainboard and 4-node cluster
by geerlingguy on Thursday August 07, 2025 @09:55PM
(#44832600)
108 GB or so under Linux.
The BIOS allows pre-allocating 96 GB max, and I'm not sure if that's the maximum for Windows, but under Linux, you can use `amdttm.pages_limit` and `amdttm.page_pool_size` [1]
[1] https://www.jeffgeerling.com/blog/2025/increasing-vram-alloc...
Re:Benchmark Framework Desktop Mainboard and 4-node cluster
by amstan on Friday August 08, 2025 @02:02AM
(#44833944)
I have been doing a couple of tests with pytorch allocations, it let me go as high as 120GB [1] (assuming the allocations were small enough) without crashing. The main limitation was mostly remaining system memory:
htpc@htpc:~% free -h
total used free shared buff/cache available
Mem: 125Gi 123Gi 920Mi 66Mi 1.6Gi 1.4Gi
Swap: 19Gi 4.0Ki 19Gi
[1] https://bpa.st/LZZQ
Re:Benchmark Framework Desktop Mainboard and 4-node cluster
by cpburns2009 on Friday August 08, 2025 @09:31AM
(#44836719)
Thanks for the correction. I was under the impression the GPU memory had to be preallocated in the BIOS, and 96 GB was the maximum number I read about.
Re:Benchmark Framework Desktop Mainboard and 4-node cluster
by sliken on Saturday August 09, 2025 @12:48AM
(#44844100)
Some older software stacks require static allocation in BIOS, but things are moving pretty quickly and allow dynamic allocation. Newer versions (or patches to) pytorch, ollama, and related, which I think might depend on a newer kernel (6.13 or so). Does seem like there's been quite a bit of progress in the last month.
Re:Benchmark Framework Desktop Mainboard and 4-node cluster
by lhl on Friday August 08, 2025 @03:37AM
(#44834539)
In Linux, you can allocate as much as you want with `ttm`:
In 4K pages for example:
options ttm pages_limit=31457280
options ttm page_pool_size=15728640
This will allow up to 120GB to be allocated and pre-allocate 60GB (you could preallocate none or all depending on your needs and fragmentation size. I believe `amdgpu.vm_fragment_size=9` (2MiB) is optimal.