You're speaking out of your behind. Of course the signs are arbitrary with respect to their signifiers, that's any language, but there are still mappings between signs and their signifiers. English is no difference, am I supposed to know what a 'm' sounds like by looking at it? Look up the Rebus principle, it happens in every language.

And with respect to the phonetic shifts, from a linguistic perspective, most changes of the phonetic radical involve one change of the initial sound, i.e. bilabial to interdental, dental to alveolar, voiced to unvoiced. With respect to the phonetic inventory of the language, these shifts are only one feature shifts within an old SPE framework. These small featural differences, whether conscious or not, (usually unconscious because we acquire languages during our infancy) are picked up and used by the speaker of the language to categorize words.