As for climate? Yes, the climate where I live (Ontario, Canada) is vastly more harsh than Rome (ancient or modern). Why do I bring this up? Because generally the people making these sorts of survivorship biased observations are also neglecting to control for other significant variables such as location.
You also didn't mention cost. Many of the great Roman structures which have survived to this day would have astronomical costs should we attempt to build them in the modern day using ancient building techniques and materials.
I'm not so convinced as far as materials are concerned. Techniques have improved and modern ones could be used (cranes etc.). The main issue I see here is that materials aren't even available anymore, nor are they legal w.r.t. building codes. So even if cost wasn't a factor, I don't see how anyone could build a modern building with Roman concrete, marble etc.