You might have an argument if there was no authentication/error-detection on the ciphertext, such that many keys would give valid decodings, and more so if it was a simple xor, such that any plain text could be a valid decoding given the appropriate key. But that's not a remotely practical cryptosystem for several reasons.