Of course these issues have always existed, but wafer costs have never been so high, and yields never so low (both because of increased defect density and worse process control with new transistors).
In addition to splitting the CPU or GPU into distinct units, you can also take other functions and use different processes for them. For example, in package I/O or L2 cache don't really see the same advantages for newer processes, so you can make these using more established (and cheaper/more available) processes.
I'm not an expert at how they combine chips. Like I said for AMD, they also wanted their units to be composable w/ small number of chips, so they basically have a die w/ a few cores and different die w/ memory, and I believe they have a proprietary communication mesh for connecting them. I think there is some considerable signal/energy overhead to communicating between chips. The cost of masks and interconnects is probably high enough to make a high number of diverse chiplets inviable, but I wouldn't say it's impossible in the future.
>I think there is some considerable signal/energy overhead to communicating between chips.
I tried looking for some info on this but couldn't find any. Do you have any source I could read on this?
Another way of looking at it is bit error rate [2].
I think you'll be hard-pressed to find concrete numbers since these designs are closely-guarded trade secrets. You might find some examples by searching for wireline transceiver / PHY papers on IEEE xplore, especially at ISSCC (conference) or in JSSCC (journal).
[1] https://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theore...
Here are a couple of articles with more detailed info:
https://semiwiki.com/semiconductor-manufacturers/tsmc/306329...
https://www.anandtech.com/show/16051/3dfabric-the-home-for-t...
https://www.eetimes.com/amd-tsmc-imec-show-their-chiplet-pla...
https://www.techpowerup.com/292256/amd-details-its-3d-v-cach...
The figures on the actual full wafer costs for TSMC 5nm are not public, but analysts say a ballpark around $17K is pretty reasonable. Using all the figures they ballpark, the manufacturing costs of the 7950X die collection, fully packaged, sounds like it averages around $70. So the cost is high as far as large scale chip manufacture goes, but this is also a part that retails for $600; there's certainly plenty of room to remain profitable at the top.
Realistically, R&D costs are the biggest consumer of profit margins still.
1: https://www.youtube.com/watch?v=oMcsW-myRCUFor example, der8auer did a test [1] with chips like the 5800X with high overclock and high voltage with stress tests for over 4000 hours and came essentially to the conclusion that the chips are likely to endure for at least 5 years under rather extreme conditions. Likely much longer under normal circumstances. The number of systems he's testing isn't statistically significant like you might get by trying this sort of thing at datacenter scale, but it is fairly illustrative.
1: https://www.youtube.com/watch?v=ZAww0c2m-ksThat last one is very significant. The hyper reflective EUV mask mirrors are incredibly hard to make and cost hundreds of millions. The mask alone no doubt raises chip cost by $10 or so.
Likewise, hundreds of engineers for 3-5 years can run up a couple billion dollars that must be recouped.
In short, if your process produces 20 defects per wafer, and you can fit 100 chip on a wafer, you're going to end up with ~84% yield(ie, slightly less than 20% loss). If you are able to split that same chip into 2 equal-sized pieces and make twice as many, your yield is now above 90%.
AMD has also made them composable in such a way that they have to produce and stock far fewer ICs to fulfill all of their SKUs, which is also another fantastic benefit.