Ask the same question: what if people don't like the local restaurant food (new chef puts too much salt) and they lost the patrons? Is that cancel culture? No. Cancel culture is when you want to close the restaurant for everyone when the owner's daughter refused to date you versus not eating there because you don't like the food, but you don't interfere with others eating there because they like the food. It's "I don't buy" vs "close them down".
If you say it's not cancel culture when a majority of people change their mind about someone, then you're saying that almost by definition cancelling is when a small minority of people have the power to deplatform someone who missteps. So, it's not accurate to say that "the left is ok with cancel culture" (for instance) since only a small group of people are going it who are not representative of the larger population.