How is this any better that "build a bot that can beat Chess Masters" ?? Well, apart from the part where this includes multiple agents, and swarm robotics, etc. But frankly, setting a game as a bar actually covers relatively less scope in my opinion. Conversational bot actually does cover a LOT of scope if you think about all the possible ways the conversation can be taken. In fact, we as humans use conversation to judge other humans' intelligence as well. E.g. job interviews.
Though perhaps an advanced turing test could include performance in a variety of social situations like...
- convince employers to hire you, - Convince a girl to go out with you - Convince a customer to buy something - Debates... (Presidential debates by AI would be interesting)
However, in this case, "Eugene" claimed English as his second language, which seems as close to cheating as it gets.
One of the ironies of AI is that trivial tasks for humans like recognizing a soccer ball, moving around without falling over, or, yes, holding a conversation are very, very difficult while difficult tasks, like playing chess or finding the best route on a map, are relatively trivial.
A better "Turing test" proposed a few years ago was, "develop a team of robots that play soccer so well they can win the World Cup". While not the best possible driver of AI research, the quest for such a goal would still drive AI research orders or magnitude better than "trick random people to mistake a chatbot for a human".