That's basically what adversarial inputs are; except that those are unintentional. A true backdoor would probably look more like a ML system trained to give a nefarious output for the "backdoor" input (but this is like poisoning), and then whatever program relies on the output of the ML system would handle that accordingly; not unlike backdoors in conventional software. The output of a neural network is only as useful or effective as the software (or person) that makes decisions based on it.