For purposes of this argument no one gives a damn about Toronto. It's another of the edge cases where attractiveness of downtown of a major city outweighs "normal" North American real estate factors.
Take a look where the most expensive houses in Hamilton or St. Catharines or Kitchener are and where the "bad" parts of town are.
For example, in Toronto the real estate prices are sky high in the city (e.g. I've heard of just a single floor of some of those large houses in The Annex going for upwards of $800k). There are a lot of condos being built (and already built), but they are all aimed at single people or childless couples, not necessarily at families. Two or three bedroom condos are few and far between, meaning that they aren't exactly cheap.