This is a bit similar to the "exploration vs. exploitation" problem [1]. Is it better to explore new avenues in hopes of higher returns later, or better to exploit current knowledge of the problem to maximize current returns?

[1] http://en.wikipedia.org/wiki/Multi-armed_bandit