Simple procedure: take a large sample (1M pictures or so), tag them with the system and then show each picture with the label to a person (using Mechanical Turk or similar) and ask "is this label offensive?"

Use simple statistics to identify trouble spots.