Inference here means "running" the model. So maybe it has a similar meaning as in econometrics?

Training is learning the weights (millions or billions of parameters) that control the model's behavior, vs inference is "running" the trained model on user data.