Okay, but let's assume you encounter a goat. It is purple. It is from Mexico. It is tiny. It seems to also be a robot. I'm guessing you've never come across someone describing such a thing to you before, such that you can identify from memory the correct way to describe that goat.

So why, when someone shouts out "Look out for that robot purple tiny Mexican goat!" you would immediately peg them as not a native English speaker, but if they had said "Look out for that tiny purple Mexican robot goat!" you would not?

This is not a 'fixed phrase' thing. There are rules, you're just not consciously aware of them.