News for hackers. Stuff that matters.

A comment on The Universal Weight Subspace Hypothesis. Parent comment

Re:The Universal Weight Subspace Hypothesis by woopsn on Tuesday December 09, 2025 @12:47AM (#46201673)
It's a little arbitrary. Look at the graph on page 6, there's no steep gap in the spectrum there. 16 just about the balance point
[ Parent | on HN ]
Re:The Universal Weight Subspace Hypothesis by moi2388 on Tuesday December 09, 2025 @05:05AM (#46203240)
But there is a steep gap in the spectrum at 16 on page 7
[ Parent | on HN ]
Re:The Universal Weight Subspace Hypothesis by yorwba on Wednesday December 10, 2025 @02:27AM (#46215078)
That's the spectrum of LoRAs, which are LoW RAnk by design.
[ Parent | on HN ]
Re:The Universal Weight Subspace Hypothesis by moi2388 on Wednesday December 10, 2025 @10:53AM (#46219183)
Yes. But from their paper: “In our analysis, we present compelling empirical evidence for the existence of universal subspaces within LoRA adapters across different modalities and tasks.”

I also don’t understand what they write under figure 2, since resnet50 has 50 layers, not 31.

[ Parent | on HN ]