Start with demographics. Layer on productivity growth and you have a first-order approximation of national productivity. Figure gross margins and from that net margins, as well as average corporate tax rates–boom, you have the economy's profitability. Estimate a pay-out rate and you get a high-level return estimate. It's very approximate, but it's theoretically solid for any closed economic system. (Cf: natural resource inputs are not properly accounted for.)
How do you explain why this appreciation in value isn't arbitraged away, as suggested by GP? In his words, "if 9.7% long-term returns were guaranteed, wouldn't everyone just borrow 30-year loans on margin at 5% interest rate and invest it in S&P?" This would effectively drive up the current day price and eliminate any projected future growth in index price -- invalidating the hypothesis that the S&P price will continue growing in the future.
I'm basically saying I'd like a somewhat bottoms up approach to modelling the S&P index where the inputs can explain the x% YoY increase in price.
I think it ultimately comes down to people trusting history to repeat itself.