The increase in productivity would have to compensate the laborers for their time. This was traditionally accomplished by using laborers whose time was of very little value. The richer society is, the more difficult this is to do.