The problem is likely the GPU price, NVidia has a very large margin - but it doesn't seem like the the use case supports it. If NVidia cut the GPU price by 2-4x then these problems likely go away.

I wouldn't be surprised if most hyperscalers are betting on cheaper GPUs on their next upgrade through either pricing pressure on NVidia, competition from AMD/Broadcom or in-house chips.