Same applies for speech models.
"generalist" multi-modal LLM mostly uses images (and videos, sounds, etc.) as input. I'm quite sure they have an output head for it, as well, used during training, but their outputs are probably not as good. They generate images much faster (hence probably cheaper), too: typically a token is 16x16 pixels, so to generate an image of modest 512x512 resolution would be the same as generating a 1024 token length string, that nowadays is pretty fast and cheap (e.g. with Sol it would be about $0.01). But the result won't be great.
Diffusion models instead are trained to generate a "denoising" step of the image. Typically they have some inputs (e.g. the owl sketch in the blog post and an embedding of your textual request), start either from pure noise or from a noisy version of an input image, and then repeatedly slightly modify it in anywhere between 4 and 100 (or more) steps (hyperparameter to be chosen, and also depends if you start from noise or from something non-pure-noise), each time making a version that the model believes to be a bit closer to where you want to get. The underlying model _could_ be similar to the language model, but I think it isn't, because (1) if you want to call it 100 times in 4 seconds it must be smaller and dedicated, and (2) it's trained to do _exactly_ that, i.e. remove 1/100th of the noise, no more, no less. So it must be a dedicated one.
A few years ago people did research on "one shot" methods (earliest were GANs of course, but also some autoregressive model like a multimodal LLM directly), but as far as I know, quality has always stayed well behind diffusion models. It was faster, though, not more expensive.
Regarding whether that's standard to pre-train multi-modal LLM, apparently that is not the case (at least, according to ChatGPT's answer, but that's reasonable - or anyway it was only a wild guess on my side). Apparently it's more common to have a text-only and an image-only pretraining phases, followed by an alignment to make the two embedding spaces a common one.
Why do you think next token prediction didn’t take off in multi modal space? That’s right! Because it’s prohibitively slow. A diffusion model, by contrast, can update an entire latent image in parallel at each denoising step. Imagine decoding images or a 10 second video one token/pixel at a time! Secondly, images are not naturally sequential. They are spatial representations of information m. Diffusion can progressively refine both local and global spatial structures in the image.
I actually think there is room for next token multi modal prediction but more where world models are concerned. Like predicting the next set of futures for a robot. But actually now I would opt for decision models instead.
But in short, your analysis, and doubts, are incorrect as to why generalist LLMs are not used instead of bespoke diffusion image gen models.