News for hackers. Stuff that matters.
Re:Baidu dropped ERNIE 5.5 Flash Base by stymaar on Thursday November 05, 2026 @08:09AM (#50378661)
A base model is what you get just from next token prediction training. Think GPT-2/3 but bigger.

Withtout RLHF, instruction-following training, or RLVR to develop reasoning.

Re:Baidu dropped ERNIE 5.5 Flash Base by otabdeveloper4 on Thursday November 05, 2026 @12:21PM (#50381732)
All LLMs are next-token predictors. The post-training is to get it to predict tokens from a structured, meaningful dialogue exchange and not from a random reddit and 4chan data dump.
Re:Baidu dropped ERNIE 5.5 Flash Base by stymaar on Thursday November 05, 2026 @04:13PM (#50384527)
Nah, that's only one part of the post training, and it has been so since OpenAI's o1 and Deepseek-R1 in fall 2024-winter 2025, a good fraction of the post training (Most maybe at this point? Who knows?) is spent on RLVR (reinforcement learning on verifiable reward), in this kind of training you don't judge the token generated themselves, but the answer. That's where all of LLMs progress over the past two years in Math and Software in software and maths come from.

So yes, they are still next token predictors at their core, but they stopped being stochastic parrots since they are now being trained on much more than just statistically plausibly imitating their training data.