News for hackers. Stuff that matters.
Re:Astra can read and write weights to improve Qwen by 3abiton on Monday November 02, 2026 @05:44PM (#50345098)
At this point, is the second L (language) a limitations to LLMs? Is reasoning in latent space directly a more efficient and more "native" inference? I remember when one researcher replicated the middle layer of deepseek model (just the "thinking" layers) and its benchmark scores got noticebly higher. All of this is moving so fast, and yet it's quite exciting.
Re:Astra can read and write weights to improve Qwen by msdz on Monday November 02, 2026 @05:50PM (#50345169)
(I don’t know anything about this field, but) I’m guessing the reason we keep language instead of going straight to latent space is that you wouldn’t want to remove the only (?) remaining concept which still somewhat allows us to peek into the “thought process” in a way we can (hope to) understand wrt. models’ interpretability…