This is word chopping, not an interesting philosophical argument. Truth is exclusionary, and the space of excluded hypotheses is at a minimum exponentially larger than the non-excluded ones, if not super-exponentially, if not some variety of simply infinitely larger, depending on how you count. Appealing to the size of the universe of false statements and/or "things you don't believe" is not meaningful.
In which case, by believing in one specific God, one is not an atheist in any sense toward other religions. Believing in one specific God literally means that you do have "belief in the existence of a supreme being or deities," your only dispute is to which one.
Another interesting question to ponder though: Who's fault is this? Is it God for not jumping out of the shadows screaming "I am he, worship me!"? Or is it an unbeliever who makes assumptions about how a God would act, and finds there is no God because he doesn't fit the unbeliever's assumptions? In which case, what is the unbeliever but a God himself?
But then again, I've never met anyone who became a believer through observing creation. Just as my confirmation bias leads me to see God in these stories, I imagine unbelievers tend to receive confirmation of the absence of God.