Keep in mind what the factories were producing - it was not luxury goods for the wealthy. It was textiles, clothing, pots, pans, all sorts of things that made life better for ordinary people.

This, in turn, is what made it possible for us today to enjoy a high standard of living unimaginable back then.