For instance, taking stuff to space is super mega expensive.
It is so expensive that I would argue it would be a lot cheaper to put solar panels into the ocean.
And since I've brought up ocean - the ocean has immense energy and if you don't have land because you are in the middle of ocean, perhaps it would be better to pursue ocean power.
[1] http://en.wikipedia.org/wiki/Geography_of_Japan#Composition_...
Factor of two would be day and night. In space, it's day all the time, while on Earth it's night half the time, on average.
However, you also have sun angle. You're not getting much energy when the sun is near the horizon. Even if your panels track the sun, there's a limit to how close to the horizon they can track before they start shadowing each other. Atmospheric attenuation also becomes extremely strong: note how you can comfortably stare at a sunset.
Atmospheric attenuation is a big factor even when the sun is overhead. Even when it's straight above, you're still losing something like 30% (again highly inaccurate, just the general idea).
And then there's weather. On cloudy days, you don't put out much. How much this affects you greatly depends on where you put your panels, obviously, but it can be a big factor.
Then you have a meta factor from the fact that these other factors vary over time. If you took a constant 2x or 6x loss on your power generation, it would be one thing. But instead, you're generating a lot of power at some times, and none at other times, and those times don't necessarily line up with demand in a nice way. So now you have to efficiently store the power when you generate it, or have lots of additional capacity to make up the shortfall at night that does nothing productive during the day.
Now, I wouldn't be surprised if Musk's overall point is correct. Flinging massive solar panels into space ain't cheap, and the money you spend on rockets could buy you a lot of solar panels in various deserts, plus high-voltage transmission lines, plus storage facilities, plus.... But it's considerably more complex than "twice the solar energy".
It means that it's possible to occupy 5x-10x more area in the space with the same amount of solar batteries as on Earth. Yes, the harvest per square meter may be smaller, due to smaller efficiency of mirror->solar panel transition, but the cost/watt may be much smaller.
Now, this doesn't refute the issues Musk is bringing up here, but he isn't taking consideration of this factor at all.