This is generally true for primary production like agriculture, however, agriculture is something less than 5% of our effort, and for the majority of our economy we have the opposite effect where economies of scale enable much greater efficiency - i.e. more or better goods/services for the same amount of labor - than it could be possible with a much smaller population.