Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on The 'Astra can read and write weights to improve Qwen' post is misleading. Parent comment
* Thinking Like Transformers” by Weiss, Goldberg & Yahav (ICML 2021).: https://arxiv.org/abs/2106.06981 * Tracr: Compiled Transformers as a Laboratory for Interpretability https://arxiv.org/abs/2301.05062 (code at https://github.com/deepmind/tracr)
So it's not really an "LLMs can modify weights" thing. Rather it's "LLMs can write programs that can write or modify weights". Indeed LLMs can write many programs, it's impressive.
There are no comments on this comment yet.
* Thinking Like Transformers” by Weiss, Goldberg & Yahav (ICML 2021).: https://arxiv.org/abs/2106.06981 * Tracr: Compiled Transformers as a Laboratory for Interpretability https://arxiv.org/abs/2301.05062 (code at https://github.com/deepmind/tracr)
So it's not really an "LLMs can modify weights" thing. Rather it's "LLMs can write programs that can write or modify weights". Indeed LLMs can write many programs, it's impressive.