News for hackers. Stuff that matters.
Re:Apptainer: Application Containers for Linux by NGRhodes on Thursday June 26, 2025 @11:14AM (#44388221)
Scale of data we see on our HPC, it is way better performance per £/$ to use Lustre mounted over fast network. Would spend far too much time shifting data otherwise. Local storage should be used for tmp and scratch purposes.
Re:Apptainer: Application Containers for Linux by snickerdoodle12 on Thursday June 26, 2025 @12:05PM (#44388707)
The docker image is a scratch purpose.
Re:Apptainer: Application Containers for Linux by trueismywork on Thursday June 26, 2025 @01:40PM (#44389545)
Imagine copying 8gb image to 96000 ranks over network
Re:Apptainer: Application Containers for Linux by 0xbadcafebee on Thursday June 26, 2025 @02:18PM (#44389888)
It's called caching layers bruv, container images do it. Plus you can stagger registries in a tiered cache per rack/cage/etc. OTOH, constantly re-copying the same executable over and over every time you execute or access it over a network filesystem wastes bandwidth and time, and a network filesystem cache is both inefficient and runs into cache invalidation issues.