>who presumably lost a large chunk of their "wealth" after the civil war

I mean, having to hire labour rather than owning slaves changes the cost structure of their businesses, but it doesn't mean their plantations suddenly stopped growing things.

If anything, this points towards the idea that slavery was economically inefficient.