Or some will just automate, while others go out of business, and then you're left with still more expensive crops than before without any workers getting paid to pick them.

But I'm sure those "slaves" will thank you.

Wall Street's interests are diametrically opposed to workers. Wall Street wants to turn good jobs into shitty jobs and ship them to more desperate poor people in poorer countries, whereas workers want the exact opposite. We've had 30 years of free trade and basically no border. A nudge in the other direction isn't going to be the end of the world. A blip on the jobs radar which comes along with stronger worker rights is gonna be okay.