I would argue that the only real way to change is attempting to effect group think. Individual purchase decisions will not do this. Only organized efforts will. Meaning everyone is just as responsible for not doing this, whether they buy the product or not.
Again, if slavery is going on, whether you buy the product involved or not, we are all equally guilty for allowing it to continue. Buying the product is irrelevent.
You claim that "rallying large groups of people is the only way to make a difference," but a large group of people is made up of individuals, and those individuals will participate in that group by altering their day to day purchases.
So to say that it is all about groups and not about any individual decision is a bit of a paradox since without any individual decision, there is no group.
The most important goal is to deeply realize your position and relation to other people - how you act after you realize this is up to you/your emotional makeup/your virtues/etc.
My claim is that because of the complicating factors (visibility and ineffectiveness) inherent in this situation you aren't able to fully understand this relationship.
The other assumption is that the only way to change this is to stop buying a product. The truth is robots are probably better than slaves. That day is coming hopefully!
There are two complicating factors here: visibility and paucity of influence. You can't see the bad thing happening and you, individually, don't have the power to stop it.
The interaction between visibility and morality is a complicated one, but the fact is that if people in the most affluent nations boycotted diamonds then diamond mining would cease. If people boycotted Coca-Cola and Fiji spring water then villagers would get their water back.
The fact that you can't control these situations also complicates things. You can stop buying Coca Cola but, unless everyone else does to, CC is just going to just keep on stealing resources.
I think that the thing I really dislike about your comment is that you have the audacity to condemn the wrongs committed, but lack the integrity to acknowledge that you play, at least, some very small part in those wrong things happening. Because you do. If you don't buy that diamond ring then that is real $$$ that the diamond miners don't get. The hole created by your boycott has a non-zero probability of shutting down a diamond mine. The fact is that you don't care. I'm not saying you should - I'm just saying you don't. And the people doing these "wrong" things don't care either. Perhaps you could say that you would do differently in their situation, but my guess is that you are REALLY not in their situation and you have no idea what their situation is AT ALL.
I get it. You're in a comfortable position. There's no reason for you to change. You probably don't care enough about people in China, etc. that you're actually going to significantly, or even insignificantly, change your day to help them out. In fact, it's not even clear that you can. This may sound like a criticism, but I really don't care what you do.
The main element missing from your comment is the answer to "what to do?" My guess is that your answer is "Nothing. There are bad people elsewhere, and I'm not to blame, so I do nothing. Let them figure it out." I guess I hope this isn't actually your answer and that you realize that it's a lot more complicated. These people are "unscrupulous" not because they're full of demons, but because globalization exerts pressure and men sacrifice their better intentions when faced with that pressure.