There have been external servants for as long as we've had nations... that's something you must get over if you're enjoying the spoils of a better off country. There's also a lot more trade going on because of better technology (seafaring, navigation, freight, etc), along with farming which is generally occupied by about 2 or so percent of the US population which was historically much higher.

Let us not forget that your home would be staffed with a small militia of servants if you had to trade back your dishwasher, washing machine, dryer, fridge, and vacuum, etc.