It depends on what you mean by “inside the box thinking”. In one sense, you are always mostly sampling high probability completions and they are going to be close to things that have been observed during training. Low probability completions will likely be low quality too. But, in a case where the context is forcing the model to generalize, I.e. interpolate in the latent space, the most probable completion may appear novel.