I stared at a partial eclipse as a child without eye protection for several periods of about 10-12 seconds (remembering that I used to occasionally look at the regular sun for 3-4 seconds without apparent injury). Unfortunately, this gave me an afterimage of the eclipse in one eye that lasted for six months (!) -- although I was fortunate that it didn't last even longer than that.

The optometrist I consulted the day after the eclipse said that there didn't appear to be significant permanent damage to my eye, but the subsequent six months of the afterimage, and not being sure whether or not it would last my whole life, were certainly no fun. Overall, I feel like I got very lucky, and I wouldn't have been surprised if the afterimage had lasted for decades rather than months. Maybe it would have if I had continued looking for just a few more seconds!

My impression is that the risk of different kinds of damage is complicated, but there are several things that can go very wrong. One is that the sun during an eclipse is much more interesting than usual, so it's so much more tempting to look longer than momentarily; another is the UV exposure that people have mentioned (where there may still be enough UV to cause damage even at times when the visible light is weak enough not to cause significant discomfort). UV is a big factor that means that we shouldn't trust our intuition about whether something may harm our eyes. In industrial settings, too, people sometimes get significant eye injuries because they're staring at UV (or IR) sources that don't subjectively feel dangerous or painful.

(Edit: taneq in this thread also mentions a problem about dynamic range where your pupillary contraction may not be enough to protect you.)

While permanent damage might not happen quite as quickly as some people may have been led to believe, and some risks may be slightly exaggerated in certain accounts, there's just no reason to take the risk of not having proper eye protection, especially when you can't really judge from the level of discomfort or lack of discomfort whether damage may be happening. As far as I know, every eclipse leads to emergency follow-up optometrist visits, and I can testify that it's no fun to be the patient in one of those visits waiting to hear the news about whether the aftereffects are likely to be permanent.