When getting down to the nuts and bolts of a Seebeck (thermoelectric) junction, the efficiency is highest when the heat flow is minimized and the temperature difference is minimized.

You get no power when those things are minimized, and in fact you want them both as high as possible.