Re:Benchmark Framework Desktop Mainboard and 4-node cluster
by amstan on Friday August 08, 2025 @02:02AM
(#44833944)
I have been doing a couple of tests with pytorch allocations, it let me go as high as 120GB [1] (assuming the allocations were small enough) without crashing. The main limitation was mostly remaining system memory:
htpc@htpc:~% free -h
total used free shared buff/cache available
Mem: 125Gi 123Gi 920Mi 66Mi 1.6Gi 1.4Gi
Swap: 19Gi 4.0Ki 19Gi
[1] https://bpa.st/LZZQ
Re:Benchmark Framework Desktop Mainboard and 4-node cluster
by cpburns2009 on Friday August 08, 2025 @09:31AM
(#44836719)
Thanks for the correction. I was under the impression the GPU memory had to be preallocated in the BIOS, and 96 GB was the maximum number I read about.
Re:Benchmark Framework Desktop Mainboard and 4-node cluster
by sliken on Saturday August 09, 2025 @12:48AM
(#44844100)
Some older software stacks require static allocation in BIOS, but things are moving pretty quickly and allow dynamic allocation. Newer versions (or patches to) pytorch, ollama, and related, which I think might depend on a newer kernel (6.13 or so). Does seem like there's been quite a bit of progress in the last month.
The BIOS allows pre-allocating 96 GB max, and I'm not sure if that's the maximum for Windows, but under Linux, you can use `amdttm.pages_limit` and `amdttm.page_pool_size` [1]
[1] https://www.jeffgeerling.com/blog/2025/increasing-vram-alloc...