Mostly heat dissipation. Assuming you hold voltage constant, heat increases linearly with frequency. Also, as process nodes get smaller, you increase the # of transistors per square inch, so a jump to a new node can also increase heat generation quadratically if all else stays constant.

As other commenters mentioned, you can counteract this with better cooling, but eventually the marginal $/watt of cooling isn't worth the tradeoff of lower speeds & multicore processors.