A few centuries ago, europeans had only ever seen white swans.

even with the obligatory "so far" tacked on the end, I think that's pretty significant

We can't know if it's significant at all based only on the fact that we haven't seen it yet. It seemed pretty significant that Europeans had only seen white swans, to the point where the concept of 'black swans' was a jest, like "when pigs fly!". Then they got to Australia and found black swans. The black swans had been there before English was even a language.

The simple fact of not having seen something is totally baseless as evidence to whether or not that something is possible.

Absence of evidence is evidence of absence. No-one has ever observed electric blue swans. How likely do you think it is that they exist? If this seems like an unfair question, that's kind of my point.