But at the same time, I find it really weird to go, "I morally can't stand for a single thing you're doing and I think you're a generally ill hearted company. Here, take more of my money to help you fund your ill-doings."
I can get behind boycotting companies we know do bad things, however I wouldn't delude myself into thinking that this is a means to ending the abuse of their workers, or a means of making the world a better place. This is a political problem, not a consumer problem.
Being an ethical consumer seems to be Sisyphean task, as to do so responsibly we'd need to ensure the alternatives we purchase from aren't treating their workers even worse to hold some market share. When we factor in the complexity of supply chains, environmental considerations, and labor practices across the globe (and the relations between core and peripheral nations,) a consumer simply cannot reasonably fit their purchasing decisions within a consistent ethical framework. This is probably the reason behind the phrase "there is no ethical consumption under capitalism."
That and boycotting is political mob mentality. Needing a random person, you'll never meet or have a coffee with, telling you what to do so you can be a "a good person" in their eyes is bullshit. Do it strictly because you believe in your own ethics.
I can think of at least some ways in which consumption might be more ethically neutral.
After a natural disaster, you often see communities band together to meet each others needs however they can. In a way, this mirrors how many native cultures approach production and resource distribution. Accepting the resources I need or providing for other's needs according to my ability certainly isn't unethical.
There is something fundamentally different about the nature of consumption in economies based on maximizing exchange value vs meeting greatest needs. Otherwise, you likely need to defend the increasingly controversial position that markets maximize utility greater than any alternative.