Edit: Okay, since some people are downvoting my joke, a more serious answer -- if your server farm is running out of space, and you're currently looking at renting additional space to accommodate your growth, EPYC Rome will take up substantially less space for a given level of performance. This is one case where it may make sense to instantly replace.
Another is to test EPYC Rome on a limited basis, to evaluate potential for a larger scale replacement in the future -- I think many companies in the space are going to fall into this camp.
Hence in the near future, there will be a modest increase EPYC Rome uptake, followed by a massive increase in the medium-term future.
If AMD's chips offer better performance/watt, that would make them attractive. The capital cost of purchasing a server tends to be a smaller number than the cost of the power to run and cool it over its lifetime.
However, what also matters is vendor support. Lots of companies have contracts for pricing and support on servers, and they won't necessarily change to save even a large amount of money on one generation of hardware. So for adoption of Zen2 CPUs in the data center, it will be critical for AMD to get the big names on board.
The next generation of the Zen chipset has already been designed.
EPYC Rome is likely being priced s.t. Xeon's would have to be sold at a loss to match $/perf.
People can't buy interesting ideas, and businesses aren't going to change system vendors overnight.
Thanks for mentioning immersion cooling as a potential solution to this; as a shameless plug, we are working very hard to make immersion cooling not exotic anymore at Submer ( https://submer.com ), solving all the problems we saw of the previous state of the art and helping with some of the biggest problems in the data center industry: cooling, power, densities, real estate costs, data center location, DC power distribution, TCO and more.
Exciting times... we are seeing a lot of trends pointing towards our immersion cooling solution and stealth traction from big names.
Is there a hypothetical point at which it becomes cheaper to spend additional capital in order to remove a perfectly-good Intel server in favor of a new AMD server (considering potential savings on cooling+power+space)?