It might not make a difference when looking at a noisy straw man #general channel with strangers, but it certainly matters when you look at everything else like PM convos, PMs you received while offline, notifications while offline, and slower or more critical channels like #staff or a group of irl friends or announcement channels or almost every other channel type I can think of outside of noisy #offtopic channels.