People keep making this argument, but forgetting that context exists in the environment as well. In the context of charades, you're playing a game. It's something that's ongoing.
Now if we took out all the stop signs and replaced them with a pause symbol with yellow and black text (thus erasing the existing contextual clues we use for stop signs), we'd see how clearly context matters to symbol recognition.
Now, your comparison to the hamburger menu does make sense, and I agree. However, the context for the hamburger menu might be "button that's alone" and it just means "menu." I can see that catching on.