News for hackers. Stuff that matters.
Re:Decomposing Language Models into Understandable Components by moralestapia on Saturday October 07, 2023 @11:20PM (#37807615)
Oh dang, I am quite literally working on this as a side project (out of mere curiosity).

Well, sort of ..., I'm refining an algo that takes several (carefully calibrated) outputs from a given LLM and infers the most plausible set of parameters behind it. I was expecting to find clusters of parameters very much alike to what you observe.

I informally call this problem inverting an LLM, and obv., it turns out to be non-trivial to solve. Not completely impossible, tho! as so far I've found some good approximations to it.

Anyway, quite an interesting read, def. will keep an eye on what you publish later on.

Also, from TFA,

>Another hypothesis is that some features are actually higher-dimensional feature manifolds which dictionary learning is approximating.

Well, it's a continuous space so you could have as many manifolds as you'd wish, so yes :^). But, pedantry off, I get the idea and that's definitely what's going on here IMO.

Re:Decomposing Language Models into Understandable Components by herodoturtle on Sunday October 08, 2023 @02:00AM (#37808191)
At first I thought this was an ode to dang.
Re:Decomposing Language Models into Understandable Components by stavros on Sunday October 08, 2023 @05:19AM (#37809024)
Oh dang, a name so spry, A clever soul, with humor wry, In life's vast game, you do not shy, A friend to all, a bond we tie.
Re:Decomposing Language Models into Understandable Components by codethief on Sunday October 08, 2023 @03:39AM (#37808589)
> One amazing realization one can get from this is, what is the conceptual equivalent of the transition functions that connect all different manifolds in this LLM space?

Could you elaborate on what you mean by "transition functions" here?

Re:Decomposing Language Models into Understandable Components by evertedsphere on Sunday October 08, 2023 @05:54AM (#37809172)
likely https://en.m.wikipedia.org/wiki/Atlas_(topology)#Transition_...

although the quoted sentence does not make sense to me; transition maps connect different patches of one manifold. it's possible the "LLM space" gp is talking about is a parameter space of some nature each of whose points is a manifold, but that seems like a stretch

Re:Decomposing Language Models into Understandable Components by moralestapia on Sunday October 08, 2023 @06:19AM (#37809260)
You're right, that's a bit ambiguous.

Instead of "all different manifolds" I should have written "all different ways to define a manifold". Also, now that I think about it, it may not necessarily be all of them.

The important thing here is that, however you define them and their transition maps, you'll find they're very much alike. As if there was some sort of general structure that is highly preferred over others ...

Re:Decomposing Language Models into Understandable Components by evertedsphere on Sunday October 08, 2023 @09:04AM (#37810090)
"if you look at the (chart) transition maps for different atlases on this manifold, they tend to look similar for different atlases"

what is the manifold here and what evidence do you have for this / what does it look like when you "define" them?

Re:Decomposing Language Models into Understandable Components by moralestapia on Sunday October 08, 2023 @10:58AM (#37811052)
Re:Decomposing Language Models into Understandable Components by codethief on Monday October 09, 2023 @07:56AM (#37819475)
Yeah, that's the definition I was thinking of, too, but the sentence didn't make sense to me, either.
Re:Decomposing Language Models into Understandable Components by LeonigMig on Sunday October 08, 2023 @06:21AM (#37809271)
Do you mean in the sense it’s hierarchical? Or am I missing the point entirely
Re:Decomposing Language Models into Understandable Components by moralestapia on Sunday October 08, 2023 @02:17PM (#37813024)
I'm not talking about that specifically but your intuition is also correct and there's a lot of research going around constructing/defining hierarchies of "learning" behavior.
Re:Decomposing Language Models into Understandable Components by bilsbie on Sunday October 08, 2023 @10:45AM (#37810925)
What was your approach to getting started doing this?

I’m curious to learn more about how LLMs work too.

Re:Decomposing Language Models into Understandable Components by moralestapia on Sunday October 08, 2023 @02:24PM (#37813081)
Try to get something like tinygrad[1] running locally, that way you can tweak things a bit, run it again and see how it performs. While doing this you'll pick up most of the concepts and get a feeling of how things work. Also, take a look at projects like llama.cpp[2], you don't have to fully understand what's going on here, though.

You may need some intermediate knowledge of linear algebra and this thing called "data science" nowadays, which is pretty much knowing how to mangle data and visualize it.

Try creating a small model on your own, it doesn't have to be super fancy just make sure it does something you want it to do. And then ... you'll probably could go on your own then.

1: https://github.com/tinygrad/tinygrad

2: https://github.com/ggerganov/llama.cpp