News for hackers. Stuff that matters.
Re:Grace Hopper, Nvidia's Halfway APU by edward28 on Friday August 09, 2024 @10:46PM (#41206942)
These CPUs are intended just to run miscellaneous tasks, such as loading AI models or running the cluster operating system. They don't need to be performant, just efficient, as the GPU does all the heavy lifting. NVIDIA also provides an option to swap the grace chips out with an x86 chip, which could deliver better performance depending on the remaining power budget though.
Re:Grace Hopper, Nvidia's Halfway APU by MobiusHorizons on Friday August 09, 2024 @11:59PM (#41207213)
If this is all there is to it, why do they have the high frequency and high l3 cache? Those seem to be optimizing for something, not just a “good enough” configuration for a part that is not the bottleneck
Re:Grace Hopper, Nvidia's Halfway APU by riotnrrd on Saturday August 10, 2024 @12:34PM (#41210596)
Data augmentation in CPU-space is often compute-light, but requires rapid access to memory. There are libraries (like NVIDIA's Dali) that can do augmentation on the GPU, but this takes up GPU resources that could be used by training. Having a multi-core CPU with fast caches is a good compromise.