Re:Building Transformers from Neurons and Astrocytes
by leokoz8 on Monday October 24, 2022 @01:28PM
(#33319649)
Hi, one of the authors here (Leo).
To clear up some confusion:
1) The paper proves an architectural equivalence between two models: neuron-astrocyte networks and Transformers. This is different from studies of representational similarity, which look at Transformer "responses" and compare them to brain responses using regression techniques.
2) The paper provides testable hypotheses for what astrocytes are doing. We can either succeed or fail at rejecting those hypotheses after comparing to real brain data. That is future work.
To clear up some confusion:
1) The paper proves an architectural equivalence between two models: neuron-astrocyte networks and Transformers. This is different from studies of representational similarity, which look at Transformer "responses" and compare them to brain responses using regression techniques.
2) The paper provides testable hypotheses for what astrocytes are doing. We can either succeed or fail at rejecting those hypotheses after comparing to real brain data. That is future work.
Happy to answer questions.
- Leo