although the quoted sentence does not make sense to me; transition maps connect different patches of one manifold. it's possible the "LLM space" gp is talking about is a parameter space of some nature each of whose points is a manifold, but that seems like a stretch
Instead of "all different manifolds" I should have written "all different ways to define a manifold". Also, now that I think about it, it may not necessarily be all of them.
The important thing here is that, however you define them and their transition maps, you'll find they're very much alike. As if there was some sort of general structure that is highly preferred over others ...
what is the manifold here and what evidence do you have for this / what does it look like when you "define" them?
They also reference, https://distill.pub/2020/circuits/equivariance/.
I'd also add, https://distill.pub/2021/multimodal-neurons/.
Could you elaborate on what you mean by "transition functions" here?