It might be possible, but nobody knows for sure, because these models are rather more mysterious than their architecture suggests.

> Maybe have the network which generates the answer be moderated by another network that assesses the truthiness of it.

Like a GAN? Sometimes you can do that, but it seems not always.

If this was simple and obvious, they'd have done it as soon as the first one was interesting-but-wrong.

Especially in a limited domain like grade school math, it seems entirely plausible that we can have models in very short order that ~never hallucinate. There's no external dependencies and the problem space is extremely well-defined and constrained. Much, much, much easier than making something like Chat-GPT never hallucinate