> For larger (amazon, google, etc) players the economic incentives against idleness are just too great.

Not all workloads are CPU-bound. Cloud providers have many servers for which the CPUs are idle most of the time, because they're disk-bound, network-bound, other-server-bound, bursty, or similar. They're going to aim to minimize the idle time, but they can't eliminate it entirely given that they have customer-defined workloads.

But if the workload is not CPU-bound then why would they care about upgrading their CPUs to more performant ones, like Ice Lake Xeons?

The workloads are determined by their customers, and customers don't always pick the exact size system they need (or there isn't always an option for the exact size system they need). The major clouds are going to upgrade and offer faster CPUs as an option, people are going to use that option, and some of their workloads will end up idling the CPU. Major cloud vendors almost certainly have statistics for "here's how much idle time we have, so here's approximately how much we'd save with lower power consumption on idle".

If the workload is PCIe I/O bound, upgrading to PCIe Gen4 would be a huge benefit.