I did a manual version of this where I played a dispatch controller in a robot, relaying inputs and outputs from GPT4, which I told was the reasoning brain in this robot. It was very remarkable to watch its train of thought in considering sensor inputs and then giving me actions to take in response.