But why is it the default? Why not match the existing tab behavior of every web browser?

Actually both IE and Firefox now (I think) insert tabs to the left. I don't know why it became a 'thing' at some point, I dislike it enormously as well.

I just tested this, and both IE9 and FF10 insert new tabs to the right in Windows 7.

I like it that way. It allows me to think of the tabs like a stack. In any case, you can easily change the default, it's not like you use a new machine every week that needs to be configured again.