This is not entirely true, in an ideal world a CMOS device just sitting there not switching would use basically no power, but in actual application there is some leakage (both form not being fully off and some current jumping from the gate). This leakage gets worse as the transistors get smaller and when you have billions of transistors even the tiniest amount of leakage adds up, the result is leakage current can be a significant amount of the power used by a modern processor.