> If you want to send an army into another country without local support you need a massive manpower and material advantage. After you account for that the two sides look more equal.

Or you simply need to be willing to sack and burn the opponent until they starve.

In addition, the slaves, who outnumbered free southerners by quite a bit, were not necessarily unwilling to cooperate with the Union soldiers. That's very different from trying to pacify an area that will fight to last man.