Could you elaborate on what you mean by "transition functions" here?
although the quoted sentence does not make sense to me; transition maps connect different patches of one manifold. it's possible the "LLM space" gp is talking about is a parameter space of some nature each of whose points is a manifold, but that seems like a stretch
Instead of "all different manifolds" I should have written "all different ways to define a manifold". Also, now that I think about it, it may not necessarily be all of them.
The important thing here is that, however you define them and their transition maps, you'll find they're very much alike. As if there was some sort of general structure that is highly preferred over others ...
what is the manifold here and what evidence do you have for this / what does it look like when you "define" them?
They also reference, https://distill.pub/2020/circuits/equivariance/.
I'd also add, https://distill.pub/2021/multimodal-neurons/.
I’m curious to learn more about how LLMs work too.
You may need some intermediate knowledge of linear algebra and this thing called "data science" nowadays, which is pretty much knowing how to mangle data and visualize it.
Try creating a small model on your own, it doesn't have to be super fancy just make sure it does something you want it to do. And then ... you'll probably could go on your own then.
Well, sort of ..., I'm refining an algo that takes several (carefully calibrated) outputs from a given LLM and infers the most plausible set of parameters behind it. I was expecting to find clusters of parameters very much alike to what you observe.
I informally call this problem inverting an LLM, and obv., it turns out to be non-trivial to solve. Not completely impossible, tho! as so far I've found some good approximations to it.
Anyway, quite an interesting read, def. will keep an eye on what you publish later on.
Also, from TFA,
>Another hypothesis is that some features are actually higher-dimensional feature manifolds which dictionary learning is approximating.
Well, it's a continuous space so you could have as many manifolds as you'd wish, so yes :^). But, pedantry off, I get the idea and that's definitely what's going on here IMO.