Most of that is due to college being an incredibly efficient social filter. The odds of a social interaction between Yale student and another being worthwhile (so to speak) are an order of magnitude higher than a Yale student and non-Yale New Haven dweller.