Re:Evidence of a predictive coding hierarchy in the human br...
by testcase_delta on Friday March 10, 2023 @02:27PM
(#35099539)
Does anyone know how this fits with (or not) Chomsky's ideas of language processing?
[ Parent
| on HN ]
Re:Evidence of a predictive coding hierarchy in the human br...
by convolvatron on Friday March 10, 2023 @03:23PM
(#35100398)
the idea that some linguistic facilities are innate? or the government binding model of grammar or something else?
for the first two, I think this orthogonal
Re:Evidence of a predictive coding hierarchy in the human br...
by IIAOPSW on Friday March 10, 2023 @07:20PM
(#35103349)
|-----long timing loop / top of parse tree-----|
| |
|-shorter / child node -| |-shorter / child node-|
| | | |
|highest freq| |highest freq| |highest freq| |highest freq|
Re:Evidence of a predictive coding hierarchy in the human br...
by politician on Friday March 10, 2023 @07:49PM
(#35103660)
Similarly, ChatGPT (gpt2-medium) has 12 heads of attention. GPT-3 has 96.
Re:Evidence of a predictive coding hierarchy in the human br...
by IIAOPSW on Friday March 10, 2023 @10:27PM
(#35104909)
I don't know how superimposed waves in finely tuned timing loops with non-linear interference translates into heads of attention, and honestly I suspect a lot of the things difficult to do with heads of attention (and other approaches of the past) come for free in a resonance based system.
Re:Evidence of a predictive coding hierarchy in the human br...
by YeGoblynQueenne on Saturday March 11, 2023 @08:50PM
(#35115464)
What are Chomsky's ideas on language processing? He's a linguist, not an NLP person.