Huge waste of CapEx for those expensive training GPUs if you can only use them 8 hours a day (same goes for all other data center hardware). Nevermind data availability and other hard problems if your DCs don't run 24/7.

You do spot pricing or preemptable instances at the margins to help maximize revenue while minimizing excess carbon.

I assume that most cloud venders are over-provisioning because their margins are so good that they'd rather overspend on inventory than miss out on revenue most of the time.

Doesn’t the data center need to run for all the AI backed services to work?

Somebody needs to run the real numbers ... what activities cost the most energy ( training to get weights, answering queries using weights, etc ), cost of types of energy in terms of external effects (fossil based energy Vs wind|sunlight) demand, bandwidth use, etc.

It's not inconceivable that there's an answer that meets needs based on only running the most energy consuming operations during daylight and migrating those operations about the globe during sunlight hours while having less demand operations run at night.

There's a real issue that needs addressing surrounding ever increasing global emissions, giving lip service while continuing to increase emission related activity is double think.