Re:The bigger-is-better approach to AI is running out of road
by pizza on Sunday June 25, 2023 @02:13PM
(#36471085)
Sounds very much like {Matilde Marcolli, Noam Chomsky, Robert Berwick}’s brand new paper on the mathematics of Syntactic Merge [0]:
workspace :: nested forests of binary trees of syntactic structures (with no label order) (= here, thoughts and meanings are assumed to be composites of mental syntax objects that can be re-combined with others. Big assumption? Maybe)
externalization process :: some mental faculty that decides the order to use when outputting thoughts into ordered strings (eg vocalization of thoughts into sentences)
The thing is, you can put a probability theory onto anything that you can count or record states of. Of course, counting the external observations may lack the richness of the internal process. I feel that this linguistic program will work its way into a lot of future LM tooling, in some incarnation or another.
workspace :: nested forests of binary trees of syntactic structures (with no label order) (= here, thoughts and meanings are assumed to be composites of mental syntax objects that can be re-combined with others. Big assumption? Maybe)
externalization process :: some mental faculty that decides the order to use when outputting thoughts into ordered strings (eg vocalization of thoughts into sentences)
The thing is, you can put a probability theory onto anything that you can count or record states of. Of course, counting the external observations may lack the richness of the internal process. I feel that this linguistic program will work its way into a lot of future LM tooling, in some incarnation or another.
[0] https://arxiv.org/abs/2305.18278