I mean, sure...there's a gray area here. I could theoretically write all of my latin characters from right-to-left, but it would be wonky and hard, if only because english is written from left to right.

I believe there's a reason for the rules and that you need to know them. I think patio11's metaphor is the correct one -- it's a lot like english spelling.