Ottawa? More of a town with lots of suburbs than a real city but dirt cheap compared to Toronto or Vancouver. I would say that it is even cheaper than Montreal.

I'm not sure about the music scene but there are a wide variety of both English and French language theatre companies. NAC, GCTC, La Nouvelle Scene, etc. I'm sure the situation is similar for Opera and Ballet.