People are working on your symbolic logic hybrid approach, but not much interesting has happened yet. This is a bit expected, since the short amount of time, and the fast scaling these language models have done in those last two years. It's been hard to outscale them with any other approach, let alone by the particular one of symbolic logic.
I think the main discripancy with particle research, is that every iteration of bigger models are also significantly better, no diminishing returns in sight.
Better for concrete applications as well. There is therefore no need to claim these models are scaled for scientific reasons in the search of AI, the accountants are happy as it is.
Our vision alone uncompressed is about 13TB per hour (!) (assuming 8k per eye resolution, "full color", and about 20hz sample rate). By the time a kid knows to play with a color book, they saw 4 PB of imagery.
You specifically mention Stable Diffusion, but SD has an analogue to the learning of an individual, textual inversion, which can adapt the NN to express new concepts with as few as a handful of examples.
It's feels a bit similar here. There are clear limitations to AI by just NNs, namely they like Stable Diffusion must be trained on so much data, far more than any single artist will ever see, and they still fall short sometimes. With NLP it seems a little more obvious because the limitations seem to lag actual speech understood by humans still, even with the massive dataset again beyond what any single human will ever digest. There are clear issues here, merely reciting to critics what your model of intelligence is doesn't address the shortcomings these systems have which any literate person can perceive. And gesturing at the data horizon (you need even more data??) is not convincing either.
Apparently this cat (the interviewed author) has some baggage, but the things he's talking about (symbols) have already been studied and used by philosophers and linguists long before AI or computers were a thing. It helps to build upon what other researchers have already done, I don't know why NN researchers don't seem to map what they are doing to existing research on intelligence in other areas, namely cognitive science. Do NN researchers have any contact with linguists or cognitive scientists generally?
Again, being a little ignorant of his baggage, I do think a hybrid approach sounds like it would be promising. Symbolic AI failed in the 70s, NN is getting a bit of the way there now and is pretty useful but misses a few things, why go all in on just one type of approach?
[0] s is basically the center of mass energy, what particle physicists mean when they say the "energy" of an interaction (collision)