The presence of a delay does not mean there is no A.I. there. Not everything is as fast as Google search, for instance IBM Watson would think about a problem for a few seconds, which is fine because it only needs to be about as fast as a human.
Bot reads "What's the temperature like near me?"
Person calls "$get user-local-temperature"
API responds "{temperature:{f:77},{c:99}}"
Human writes "It's 77 degrees outside!"
Training set now contains that relationship between that question, that API call, that response, and that natural language response (and probably the users location, age, gender, and so on, all captured in the meta-data about the response in the corpus). Bot reads "What's it like outside?"
Person calls "$get user-local-weather"
API responds "{weather:{now:Sunny},{today:Cold}}"
Human writes "It's sunny now, but will be cold later today."
And so on. I think the goal here is training on standard API calls as the response, and taking their data return and converting it into grammatical sentences. It's a two step training process. Know which API to call, and know how to convert API response to natural language.There's no serious corpus yet for that -- if this is real, it is important work.
It doesn't need to see "What|WP 's|VBZ it|PRP like|IN outside|IN ?|."
Just "What's it like outside?", and know that it will always respond to that with the same call.
Warning: This is mere conjecture.
The data around how users interact will also be important:
Do they prefer a back and forth conversation, or do they want to say everything in one go.
Do they want to start a conversation, drop it, come back to it several hours later, or do they like completing it in one go.
How do they handle switching back and forth between different contexts, if certain requests take time, or do users not switch context.
What data are they happy to share, and what are they not.
What are the typical response times that a user considers acceptable 5 seconds, 1 minute, 5 minutes, 60 minutes? Does it vary depending on scenario.
Is there particular services or information that there is a trend towards, for example local search requests, research/information, particular types of purchase etc.
We've just spent 6 months going through a very similar process to this, which has helped drive the development of our Converse platform, which allows people to build semi or fully automated conversational messaging services, so this is fairly closely related.
(From our point of view, the NLP data we gathered was useful, but it wasn't the most important part)
The implication to me is that the chat is with a human, who is using an AI tool with the intention of training that tool. What better way to train a new service than to launch it, then answer all the weird, unexpected questions with humans? Gradually more of the questions get answered, the AI gets better trained, and the human-AI becomes increasing more AI.
Further, as the AI gets better, the human working with it has to do less, so they can roll out the service to more users without requiring more staff. Perhaps eventually, no human is needed.