Another layer to this is that numbers one-off the factors have nice patterns too. For base 10 those are (2, to include the factors), 3, 4, (5), 6, and 9 sort-of, since it's one below 10. Just think how awkward 7 and 8 times tables were compared to the rest. With base 12 I found that all numbers, even 7 and 11, end up having usable patterns and are easier to count by. Of course, I'm still not used to having 12 digits, but on paper I could tell they would be pleasant to count by if I had learned base 12.

And an even deeper insight is that it doesn't really matter that much and isn't worth shaking up the whole world to change. We're not going to be better or worse at math because of our number base.