There's a third option. To some people the word shit is just a word when used as a word and no more. To others it can invoke strong and vivid visual imagery, and by that, provoke a strong discomfort and distaste (as it happens I'm somewhere in the middle). So it could be down simply to that in which case it's just consideration for other people's feelings.

I too prefer colloquialisms and straightforward speaking, and I swear plenty, but if it makes someone uncomfortable, that could be considered bad manners.