The word "model" is another of those words that Minsky called "suitcase words"--- they can be used to mean many things. I don't think Marcus is saying that that LLMs have "no model", just that they don't have a model of the type that a symbolic system could have. He gives many examples of deductions that a symbolic AI system can easily do, which GPT-3 is simply incapable of.
It seems clear to me that if the CLIP-like part of Imagen or Stable Diffusion can take an image made of pixels and yield "cat" and similarly take "dog" and produce a 3D neural radiance field that we can light just like any other 3D model and recognize as a dog then there must be an accurate and useful model of both how vision works and what dogs look like and the English relationship between those two things inside the machine.
I also wish Gary Marcus was replying to Google's Minerva paper instead of GPT-3. The ability to answer multi-step symbolic problems is basically here. https://ai.googleblog.com/2022/06/minerva-solving-quantitati...
I'd also note that decades of attempts at automated theorem proving with symbolic systems haven't yielded similarly impressive results. Now we have deep learning models helping with proof search.
It's feels a bit similar here. There are clear limitations to AI by just NNs, namely they like Stable Diffusion must be trained on so much data, far more than any single artist will ever see, and they still fall short sometimes. With NLP it seems a little more obvious because the limitations seem to lag actual speech understood by humans still, even with the massive dataset again beyond what any single human will ever digest. There are clear issues here, merely reciting to critics what your model of intelligence is doesn't address the shortcomings these systems have which any literate person can perceive. And gesturing at the data horizon (you need even more data??) is not convincing either.
Apparently this cat (the interviewed author) has some baggage, but the things he's talking about (symbols) have already been studied and used by philosophers and linguists long before AI or computers were a thing. It helps to build upon what other researchers have already done, I don't know why NN researchers don't seem to map what they are doing to existing research on intelligence in other areas, namely cognitive science. Do NN researchers have any contact with linguists or cognitive scientists generally?
Again, being a little ignorant of his baggage, I do think a hybrid approach sounds like it would be promising. Symbolic AI failed in the 70s, NN is getting a bit of the way there now and is pretty useful but misses a few things, why go all in on just one type of approach?
[0] s is basically the center of mass energy, what particle physicists mean when they say the "energy" of an interaction (collision)
People are working on your symbolic logic hybrid approach, but not much interesting has happened yet. This is a bit expected, since the short amount of time, and the fast scaling these language models have done in those last two years. It's been hard to outscale them with any other approach, let alone by the particular one of symbolic logic.
I think the main discripancy with particle research, is that every iteration of bigger models are also significantly better, no diminishing returns in sight.
Better for concrete applications as well. There is therefore no need to claim these models are scaled for scientific reasons in the search of AI, the accountants are happy as it is.
Our vision alone uncompressed is about 13TB per hour (!) (assuming 8k per eye resolution, "full color", and about 20hz sample rate). By the time a kid knows to play with a color book, they saw 4 PB of imagery.
You specifically mention Stable Diffusion, but SD has an analogue to the learning of an individual, textual inversion, which can adapt the NN to express new concepts with as few as a handful of examples.
People who don't believe the LLMs have a world model are wrong because they are mistaking a physically weak world model for no world model. GPT-3 doesn't understand physics well enough to embed models of the referents of language into a unified model that has accurate gravity and motion dynamics, so it maintains a much more dreamlike model where objects exist in scenes and have relationships to each other but those relationships are governed by literary relationships instead of physical ones and so contradictions and superpositions and causality violations are allowed in the model. As multimodal transformers like Gato get trained on more combined language and sensory input their world models will become much more physically and causally accurate which will be reflected in their accuracy on NLP tasks.