Re:Intel discontinues pathfinder for RISC-V program
by imtringued on Saturday January 28, 2023 @09:53AM
(#34557978)
Intel could simply invest in sublinear deep learning.
CPU have a significant advantage over GPUs that they have more main memory. The larger your model is, and the smaller the memory of each individual compute node is, the more time is spent moving data around.
You might object that dense machine learning is mostly compute bound on CPUs but that isn't true. You can use sparse learning algorithms with sublinear complexity. In fact, the papers often show something extremely counter intuitive. The performance is memory bound at low thread counts and is compute bound at high thread counts!
I don't know the details but I suspect that when you have sparse data, your cores spend a significant amount of the time chasing pointers and this means as you add more cores, more and more of the pointers you are chasing have already been loaded into the cache by a different core.
One problem with this strategy is that Intel has super linear pricing on their high core count CPUs. The 24 core CPUs might be cheaper than an Nvidia GPU but the 40 core CPUs cost 1x to 3x as much as Nvidia GPUs depending on whether you want a single, dual or quad socket setup and in this case you obviously always want to go quad socket so intel may not be cost effective unless you want to train models that need TBs of RAM.
CPU have a significant advantage over GPUs that they have more main memory. The larger your model is, and the smaller the memory of each individual compute node is, the more time is spent moving data around.
You might object that dense machine learning is mostly compute bound on CPUs but that isn't true. You can use sparse learning algorithms with sublinear complexity. In fact, the papers often show something extremely counter intuitive. The performance is memory bound at low thread counts and is compute bound at high thread counts!
I don't know the details but I suspect that when you have sparse data, your cores spend a significant amount of the time chasing pointers and this means as you add more cores, more and more of the pointers you are chasing have already been loaded into the cache by a different core.
One problem with this strategy is that Intel has super linear pricing on their high core count CPUs. The 24 core CPUs might be cheaper than an Nvidia GPU but the 40 core CPUs cost 1x to 3x as much as Nvidia GPUs depending on whether you want a single, dual or quad socket setup and in this case you obviously always want to go quad socket so intel may not be cost effective unless you want to train models that need TBs of RAM.