Whose theory are you describing in the first paragraph? Yours? In practice, the needs of society -- new freely available knowledge -- match up imperfectly with the needs of the people who can or should do the research: money, recognition by their peers, time to spend on fun problems, and general satisfaction (which can and almost always does come from a large degree from learning something new that other people will want to learn too).