But we really need to get down to <$10/kg. Thing is, that's entirely achievable. I believe the ultimate future here will be orbital rings [1]. Space elevators get a lot more attention and they really shouldn't because they're a lot less achievable and they require materials we haven't invented yet (to resist the centrifugal force).
Imagine being able to take a cable car into orbit. That's what orbital rings promise and you need little more than copper wire and stainless steel.
Not only would this bootstrap colonization of space but you can simply attach collectors to the ring itself and run the power down a cable to the ground so you don't even need to suffer the power loss from wireless transmission (which, for the record, is a practical method still).
Isn't this the same capital expenditure analysis your post starts with though? How many billions does it cost to get the solar panels to orbit in sufficient quantity? And ground stations to receive the energy beams (microwave presumably). This is where Musk/Spacex push for cheap kg to orbit really matter. Even in the 70's they worked out that mining the moon for raw materials to build space based solar was much more economical.