I don't think imaginary internet points serve entirely or even mostly to punish.

It served to highlight valuable content and in some venues to weed out low value content entirely. This is more significant in large bodies of comments. Example highlighting the most useful 20 comments out of 700.

Interestingly sometimes wrong information can lead to good threads where people provide helpful corrections. In this case the thread is valuable for stirring up discussion even if the information may be wrong.

An additional point it may be sometimes be more worthwhile to downvote incorrect information only if it seems unlikely to bear useful fruit and after providing enlightening information to the commentor if possible.