Imagine if Australia bookstores followed the principle that even if a minority (and in this case, a very important minority as the aboriginal people are the "original owners" of the land) does not like a book, it should not be made available... you wouldn't be allowed to sell any books that contained or even mentioned deceased people.
But now, imagine that the British had not invaded and forcibly taken over Australia, so that the aboriginal people would be the current rules of the country. Now, you would probably expect no bookstores to want to offend their customers, so they would, presumably, avoid selling most books, arguably.
So, yes, I think that whether or not it's a minority that takes a certain stand makes a whole world of difference.
[1] https://en.wikipedia.org/wiki/Australian_Aboriginal_avoidanc...
Maybe it's more important to look at the overall impact on society as it is right now, and in the context of where we want it to evolve, rather than what particular individuals may like.