Decomposing Language Models into Understandable Components

Re:Decomposing Language Models into Understandable Components by adamnemecek on Sunday October 08, 2023 @12:17AM (#37807828)

All machine learning is just renormalization which in turn is a convolution in Hopf algebra. That's why you see superposition

"In physics, wherever there is a linear system with a "superposition principle", a convolution operation makes an appearance."

I'm working this out in more details but it is uncanny how much it works out.

I have a discord if you want to discuss this further

[ Parent | on HN ]

Re:Decomposing Language Models into Understandable Components by esafak on Sunday October 08, 2023 @01:19AM (#37808053)

Do you mean all ML or just large neural networks? Where is renormalization in a tree model? What superposition are you referring to?

[ Parent | on HN ]

Re:Decomposing Language Models into Understandable Components by adamnemecek on Sunday October 08, 2023 @11:59AM (#37811626)

Renormalization is all about this symmetric partitioning.

[ Parent | on HN ]

Re:Decomposing Language Models into Understandable Components by LeonigMig on Sunday October 08, 2023 @06:34AM (#37809331)

I suppose we should be cautious, the human mind is capable of overfitting too

[ Parent | on HN ]

Re:Decomposing Language Models into Understandable Components by adamnemecek on Sunday October 08, 2023 @11:48AM (#37811530)

You have no clue what you are talking about.

[ Parent | on HN ]