I'm not saying that they reject the possibility that they might be wrong; they couldn't do that any more than they could reject the possibility that they're wrong about anything immeasurable (to themselves).
The very definition of Athiesm is to deny the existence of a god or a higher power.
As soon as you admit there is a real possibility that there may be a god or a higher power; bam, you're an agnostic.
Which is the crux of my point - the definition of Atheist is often misused, as in this article. There are far more Agnostics than Atheists, it's just that Atheism gets used more frequently.
This page discussing the differences far more eloquently than I ever could:
In fact, I'm pretty sure not a single person reading HN is an atheist by your definition. Even Richard Dawkins is an agnostic by your definition. I can only conclude that your definition does't make sense.
Challenge: find me an atheist who fits your definition.
You'll find none.