Something I've been thinking a lot about lately is the potential historical usefulness of LLMs which have been selectively trained on material from before a certain date, presumably resulting in a model exhibiting attitudes of the time in question. It would be extremely interesting to be able to ask for a 1950s take on a modern idea.

For this, large amounts of very old text would be extremely valuable.