Yeah, but then the question arises: why? Assuming 4x $700 for a mini PC setup, you still only get like 360GB/s memory bandwidth. It is a lot of money with nothing to show for it, really. Your model will run at 2 tokens per second. A GPU setup within the same budget could run a 70B model at high speeds.
This is why I am kinda bullish on the XDNA NPU from AMD. It represents the low end. People who run small models below 16GB in size. Since the hardware will be available on almost every AMD laptop, it will be much easier to actually utilize in say a video game. As of today, it is kind of annoying when an LLM based game asks you for an API key. The out of the box experience isn't that good.
The bandwidth per core is the same for a 16-core Ryzen and for a 96-core Epyc.
Of course if you compare it with a GPU, the GPU wins in memory bandwidth. However the GPUs are useless for FP64 computations, unless you are a big corporation that can afford the huge prices of the "datacenter" GPUs.
For ML/AI, GPUs are the normal choice, except when you cannot afford those with enough memory, when you may still want to fall back to cheap CPUs, as argued in the parent article.
A $1500 Ryzen 9 9950X system will have a double price in comparison with a mini-PC, but it will be 3 to 4 times faster in any program whose performance is dominated by the speed of the array operations. This speed ratio is true for the fastest CPU that could be put now in a mini-PC, i.e. AMD Strix Point (a HX 370 does 192 FP32 FMA per clock cycle, while a 9950X does 512 FP32 FMA per clock cycle and its clock frequency is much higher; it should be noted that these values are similar to those for integrated GPUs; the best iGPUs have a double number of ALUs, but their clock frequency is much lower, slightly more than half of that of a CPU). In comparison with a low-power Intel CPU or older AMD CPU, the speed advantage of 9950X would be even greater.
I actually happen to have in my home a cluster of five eATX cases, which stay on two adjacent IKEA metallic tables and which are connected in a ring of direct 10 Gb/s Ethernet links (i.e. they all have dual-port NICs). Therefore it is not really impractical, even if such a configuration may be infrequent.
In the distant past, I was using dual-socket Xeon or Opteron motherboards in those cases, because such MBs and CPUs were much cheaper than today. Then, by the time of Zen 1, when the Epyc CPUs were still very cheap, the upgrades have been to single-socket Epyc MBs. More recently, Epyc became much more expensive and Ryzen CPUs have much better performance per dollar, so the latest upgrades have been to Ryzen MBs. I look forward to the launch of Ryzen 9 9950X, which should double the throughput of the vector operations, 5 years after Ryzen 9 3950X (Zen 2) has done the same in 2019.
As my PC, I use an Intel NUC. Whenever I need to execute something for which that would be too weak, I launch it on one or more of the servers, by using Wake-on-LAN and shutting them down after the task is completed. In this way, my average electrical power consumption is much less than if I used a beefy desktop, while the peak performance is much higher.