marks the date
The structure of that test (this was a while back - couldn't do an unconstrained one) had a panel composed of some chat bots and some humans (10 total, though the number isn't too important). Each was limited to the area of their specialty - I think there was a chatbot that had its specialty of bartending drinks.
The people evaluating them rated them on a distinct 1-10(?) scale. The question was trying to find a "what is the most human like chat bot."
Part of it was that the human expert in Shakespeare was considered to be more computer like than many of the chatbots in part because of the depth of knowledge in that domain.
https://github.com/ishan0102/aoc-2022-chatgpt
> Day 5: Supply Stacks
> This day actually threw me for a loop because parsing the input is so challenging. After spending a while on prompting I wasn't able to produce anything meaningful. I might actually stop after today if the puzzles keep getting harder because I don't want to spend all this time messing with input parsing. This highlights some of the limitations well, we still need human input to coax the model into understanding how to break down a hard problem.
For that repo (not mine - mine is over at https://github.com/shagie/AoC_2022 ), day 5 is where they got to the point where the complexity of the description of the problem is greater than the complexity of the problem itself.