(crustaceans are a good example as traces of them tend to get included into algal based food, but less pleasant sounding examples are e.g. from time to time small worms being in fruited pressed for juice, sound terrible but isn't really that terrible and impossible to avoid without using pesticides which are worse then that by far)
Except that fish are much more closely related to mammals than crustaceans are to insects (same phylum, different class). Here is an example of an animal that is as closely related to a cow as a grasshopper is to a lobster: https://en.m.wikipedia.org/wiki/Salp
Edit: Even more, dietary preferences tend to focus on a single species. Plenty of people eat chicken but not turkey, or eat pork but not boar. There are even people who eat lamb but not adult sheep. And no one eats wolf or eagle meat. To then come and say "hey, you already eat one arthropod, why not another" is pretty absurd.
I'm not at all against eating insects, I'm just pointing out that the fact we eat shrimp has 0 to do with it.
Either way, while we eat some mammals and birds, we still don't eat the vast majority of mammals or birds, and we barely eat any reptiles or amphibians or fish.