It doesn't specify _only_ one colour ("each card is plain white except for" would appear to work well). If your question is going to be sloppy then you're going to need to be prepared to accept a range of solutions.

There's a difference between your examples and the question:

"Give me a shirt." is a request for one shirt.

"You've got a shirt in the wardrobe." is a statement that doesn't tell you how many shirts are in the wardrobe, just that there is one.

Similarly:

"Will you hand me a pencil." is asking for one pencil.

"I have a pencil." is telling you that I have at least one pencil.

Based on en-gb native use.