Exactly. It's a fix that doesn't work, apparently, so that's why I'm thinking towards the runtime.
> it would also multiply the model runtime.
Predictably so, I would think? Such an approach could scale decently since it's not adding a dimension to the runtime, just a multiple.
More problematic is that your approach isn't going to actually work, since CNNs are just too flexible (they can learn even completely random labels) and only generalize by accident. No input augmentation technique that doesn't cover every possible modification is going to be robust against adversarial examples, and getting that amount of coverage requires an exponential blowup in runtime. The adversary has the advantage of being able to choose one modification, while the model needs to defend against all of them.
While that would likely improve results a bit, it would also multiply the model runtime. That's why the other replies directly jump to talking about training data augmentation, since that can give you similar benefits without the runtime penalty.
However, random augmentation can't fully protect against adversarial examples. The number of input variables is simply too large, and there are exponentially many directions in which they could be modified. Data augmentation can't cover all of them, and a single modification that confuses the model slightly can be amplified into an adversarial example that causes a total misclassification.