the observation goes beyond garbage in garbage out. Mainly that we're always operating from some prior and limited understanding. That what may look like a hallucination could be closer to the truth than our current frameworks of understanding allow us to admit. The hermeneutic circle.

Interesting. I wonder if current LLMs can break out of human limitations and understand the world more correctly.

A properly designed benchmark won't use tests that leave room for ambiguous interpretation.