If you limit yourself to concepts you can express by relating them to the physical-world objects and human actions then you lack the language to express abstract concepts entirely. The words used by mathematicians are chosen to describe these structures have no real-world connotations because the ideas, in their-full generality, have no representation in our every-day experience. There's no way you'll ever be able to describe any reasonably complex structure from abstract algebra by concatenating a bunch of household adjectives, it doesn't capture the full-generality or convey any more information.