News for hackers. Stuff that matters.

A comment on Astra can read and write weights to improve Qwen. Parent comment

Re:Astra can read and write weights to improve Qwen by Vaushite on Monday November 02, 2026 @08:03PM (#50346623)
Even with PRH you can apply orthogonal matrices to intermediate vectors to generate different weights with similar effects.

Even if a slight rotation passing through a nonlinearity isn't good enough, you can just reorder the vector components which makes literally no difference.

[ Parent | on HN ]
Re:Astra can read and write weights to improve Qwen by ACCount39 on Monday November 02, 2026 @08:20PM (#50346766)
Representational similarity analysis is concerned with representation structure independent from the coordinate system it sits in. A map rotated 45 degrees is still the same map.

A "reorder the vector components" transform preserves the similarity perfectly. So does a transform that maintains orthogonal equivalence.

[ Parent | on HN ]