You're replying to a comment that already answers your question. :-) I'd use slightly different words and say:

• When you work modulo 4 (as in the article in the context where this comes up), there are three kinds of primes: 2 is a solitary outlier, and all other primes are equally distributed between being 1 and 3 mod 4.

• When you work modulo 10, there are six kinds of primes: 2 and 5 are outliers, and all other primes are equally distributed between being 1, 3, 7 and 9 mod 10.

So 2 is an outlier when working mod 4 (or any even number). Mod 4 is crucially important in the context of quadratic reciprocity, so it matters here.