I'm not going to hype the technology more than it needs, but it seems to do a pretty damn good job of just this. And to the author's point, it does get this right.
And in fact, I'd suspect many of the cases where the LLM gets this wrong are caused by lots of examples in the training data which were written at different times, where someone else was chancellor. Or where the LLM gets numbering wrong (which is a well established problem).
And this is just one thing that makes sorting out what's going on here difficult - essentially, we have biologically LLM's [thinking they are] decoding silicon LLM's, of course the results are going to be weird and counterintuitive.
[1] Examine the specific language you are using, but also the colloquial ~interpreted/virtualized assertion that one is left with, if they are reading your comment "in good faith".
They implicitly recognize (and model) patterns in text, then continue those patterns.
While the patterns that an LLM has modeled may align to the high abstraction we call language, LLMs actually work at a much lower level of abstraction: plain text.
It's the content of that text that is significant. We encode language into text, and LLMs infer something "close enough" to language by blindly modeling it.
The fact that the current ones don't is surprising.
This problem exists for both LLM's and humans, I think it may be fundamental to reality itself, as it currently is at least.
I wonder if they changed the training data to consistently use "equals" where possible (even if it sounds weird to the reader) would make a difference.