It reminds me of something I do in my current occupation, with a robot. It makes predictions about how a certain motor movement will result in a change in position, then measures progress in the real world. Should the prediction and reality diverge significantly (more than a particular amount), it stops and calls a human to look into what happened. I can imagine AI techniques using this kind of thing to guess what will happen next in a series of events, with the result's divergance from the guess weighting the learning rather than a steadily decreasing weight. Perhaps this is already a thing though, I'm no AI researcher

i do not believe that young children explicitly/consciously simulate in their minds to learn cause and effect and other things, though this might be up for debate

There's things like metaphors, slang, topical references, all things that end up in conversations that change regularly in their meaning and usage. Then there's the unspoken parts of conversations. The implications behind different wordings, ways of writing things, words left unsaid, sarcasm, tone(even in text tone is conveyed in some form).

Humans do all these things while conversing almost subconsciously, each individual person does these things and comprehends these things uniquely based on years and years of sensory input and model building from a huge variety of sources, in a way that's individually unique and poorly understood. How would you even begin to train a model to even a remote semblance of that?

The gulf between even the most disparate human tribes is nothing compared to the gulf between an AI chat-bot and a human being.

I think it's entirely possible to make bots with simple models of the world. (Not that this is really a move any closer to that)

But that is basically what humans do - we don't run internal simulations, we have a model we've trained over a lifetime of experiences with the world that judges the likelihood of a given scenario

(Reading TheBookOfNow and Causality in parallel)

I think we're trying to stretch too far here. We've always dreamt of having a conversation with a bot without actually constructing the intelligence behind that bot. And it's not going to work.

We should start with something simpler, like cats, which definitely have feeling, but have a mental capacity that is closer to what we may be able to emulate through software. And then monitor progress through its 'meowing', 'purring' and 'tail wagging'.

Well, at least the ML people have.

This is automated nattering. These systems have no clue, but pretend they do. It's autocomplete with a big data set. The illusion falls apart after about three sentences or interactions. That's not much better than classic Eliza.

Most of the systems that can actually answer questions have some notion of "slots". As in, you ask "How do I get to X", and that matches something like "How do I get to DESTINATION from STARTINGPOINT by TRANSPORTATIONMODE". There are limited systems like that, slightly smarter than phone trees. You can get something done them in their limited area.

MIT has the START system which does something like that. Stanford has the SQUAD database. Both draw heavily from Wikipedia. I gather Alexa works something like that when ordering, using the Amazon catalog. Eliza had a few "slots"; if you revealed your name, or the name of wife/mother/siblings, those were stored for later template instantiation. It's not much of a mental model, but it's something.

Few of these systems have enough smarts that if you ask a question that needs clarification, they ask you a question to get the data they need. "How do I get to Pittsburgh?" ought to come back with "Where are you", or "Pittsburgh, PA, or Pittsburgh, CA", to fill in the slots. Without someone having to make up a menu tree for that class of question.

Like if you're going to create a creature from scratch why create something that can't communicate with you?

Unless you mean we should just tackle simpler forms of intelligence in general, which is what the rest of the AI field outside of conversational AI are doing.

The question is whether this approach will ever confer the ability to generate new responses, or is just building a big lookup table already encodes a potential response to any input.

Such models have to make generalizations to fit into mere 2.6 billion parameters as ten word sentence presents a possibility of one billion billion different inputs.

Generalizations are limited, of course. Arithmetic doesn't seem to be generalized, for example, which isn't that surprising given that people need around 4 years to be taught to do it (as opposed to learning it by analyzing random texts), and then they usually need pen and paper in addition to their 100 billion neurons to do non-trivial calculations.

I'm not saying that current network architectures are capable of all generalization humans can do. And we can't teach them like we do with children.