That only happens for, what, less than a second before your eye adjusts? It's not like your eye knows it's day or night, and the pupil stays fixed.