> our retinas won't last long
I think you are mistaking cause end effect. If the sun were a point source, the layout of our eyes (and our behaviours) would have evolved differently to compensate.
Alternatively, if the diameter of the sun (or the apparent diameter of the sun) quickly changed that much, our eyesight is the least of our worries.
But indeed, not all stars are "point light": the sun as seen from the earth is not. And it is a good thing, because otherwise that would make our sun like a laser, our retinas won't last long...