Easy is relative to the experience. If we're talking (26+26+10)^100 versus (26+26+10+10)^100, the difference is moot in terms of brute-forcing the password. However, the inability to double click select, to paste in to chat rooms, etc may change based on which characters are allowed. These properties may be desired (or not) for your organization.