News for hackers. Stuff that matters.
Re:Evidence of a predictive coding hierarchy in the human br... by IIAOPSW on Friday March 10, 2023 @07:20PM (#35103349)

    |-----long timing loop / top of parse tree-----|
    |                                              |
    |-shorter / child node -|                      |-shorter / child node-|
    |                       |                      |                      |
    |highest freq|          |highest freq|         |highest freq|         |highest freq|
Re:Evidence of a predictive coding hierarchy in the human br... by politician on Friday March 10, 2023 @07:49PM (#35103660)
Similarly, ChatGPT (gpt2-medium) has 12 heads of attention. GPT-3 has 96.
Re:Evidence of a predictive coding hierarchy in the human br... by IIAOPSW on Friday March 10, 2023 @10:27PM (#35104909)
I don't know how superimposed waves in finely tuned timing loops with non-linear interference translates into heads of attention, and honestly I suspect a lot of the things difficult to do with heads of attention (and other approaches of the past) come for free in a resonance based system.