Haha this scenario is perfect and tells you everything you need to know about the state of AI.

We’re sitting here wondering if the language model also models reality, when we can just ask the thing itself!

We are living in an Asimov novel