Yea makes me wonder what data could be used to train this theoretical symbolic model that precedes the language model?

I’m thinking training on movies/tv has the physical element that pure text is missing.