GPT does nothing but predict the next word (token, to be more exact). If you ask it for hints, it's going to come up with some hints that sound like they could be plausible in context of the original riddle. If you then ask it what the answer was, it's going to try to take all the earlier hints into account, because that's now the text it's trying to complete.

There's no entity sitting on the other side of the screen thinking about riddle's solution from the start. There's just a model that completes the given text one token at a time. You should expect similar results as if it was bunch of people passing a single piece of paper to each other, each writing a single word on it, not communicating with each other in any other way and trying their best to come up with coherent result.