Re:Can we do better than R-squared?
by ajtulloch on Tuesday May 27, 2014 @04:41AM
(#7803513)
> I realize this is the typical mathematician argument.
Interestingly, a mathematician's argument on this exact method was actually given in 1977, in "An Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaike's Criterion". [1]
What Stone showed was that (modulo technicalities) performing this procedure - leave one out cross validation, the procedure you claim "cannot work correctly" - was asymptotically equivalent to minimising the AIC [2] of the model. So we have an asymptotic guarantee (for whatever that's worth) on the behaviour of this procedure.
Interestingly, a mathematician's argument on this exact method was actually given in 1977, in "An Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaike's Criterion". [1]
What Stone showed was that (modulo technicalities) performing this procedure - leave one out cross validation, the procedure you claim "cannot work correctly" - was asymptotically equivalent to minimising the AIC [2] of the model. So we have an asymptotic guarantee (for whatever that's worth) on the behaviour of this procedure.
[1]: http://www.jstor.org/stable/2984877
[2]: http://en.wikipedia.org/wiki/Akaike_information_criterion