I feel like embeddings will be more powerful for understanding high dimensional physics than language because chaotic system predictability is limited by its compressability. Therefore an embedding is able to capture how exactly compressible the system is and therefore can extend the predictability as far as possible.