Probably the easiest way to describe an LLM that it's a policy. There is a reason that word has stuck in RL.

And it's not just RLVR. RLHF has been going on for years and years. LLMs have not been "next token predictors" for probably 5-6 years.