AIC is an estimate of prediction error. I would caution against using it for selecting a model for the purpose of inference of e.g. population parameters from some dataset (without producing some additional justification that this is a sensible thing to do). Also, uncertainty quantification after data-dependent model selection can be tricky.
Best practice (as I understand it) is to fix the model ahead of time, before seeing the data, if possible (as in a randomized controlled trial of a new medicine, etc.).
Re:P < 0.05 Considered Harmful
by ramblenode on Monday April 10, 2023 @08:26PM
(#35519151)
And it is not uncommon that an intentionally bad model (low AIC) will be used for inference on a parameter when one wants to test the robustness of the parameter to covariates.
p-values were created to facilitate rigorous inference with minimal computation, which was the norm during the first half of the 20th century. For those who work on a frequentist framework, inference should be done using a likelihood-based approach plus model selection, e.g. AIC. It's makes it much harder to lie.