News for hackers. Stuff that matters.
Re:Decomposing Language Models into Understandable Components by evertedsphere on Sunday October 08, 2023 @05:54AM (#37809172)
likely https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_...

although the quoted sentence does not make sense to me; transition maps connect different patches of one manifold. it's possible the "LLM space" gp is talking about is a parameter space of some nature each of whose points is a manifold, but that seems like a stretch

Re:Decomposing Language Models into Understandable Components by moralestapia on Sunday October 08, 2023 @06:19AM (#37809260)
You're right, that's a bit ambiguous.

Instead of "all different manifolds" I should have written "all different ways to define a manifold". Also, now that I think about it, it may not necessarily be all of them.

The important thing here is that, however you define them and their transition maps, you'll find they're very much alike. As if there was some sort of general structure that is highly preferred over others ...

Re:Decomposing Language Models into Understandable Components by evertedsphere on Sunday October 08, 2023 @09:04AM (#37810090)
"if you look at the (chart) transition maps for different atlases on this manifold, they tend to look similar for different atlases"

what is the manifold here and what evidence do you have for this / what does it look like when you "define" them?

Re:Decomposing Language Models into Understandable Components by moralestapia on Sunday October 08, 2023 @10:58AM (#37811052)
Re:Decomposing Language Models into Understandable Components by codethief on Monday October 09, 2023 @07:56AM (#37819475)
Yeah, that's the definition I was thinking of, too, but the sentence didn't make sense to me, either.