I think many traffic signs are visual. Ie they would not work as well if you did random assignment of sign graphics vs its purpose. Not all of them are, of course.

For a very basic and obvious example, possibility to turn right vs turn left signs look different not just because of convention but they resemble the shape of the road.