This is basically adversarial training, which is a typical (& very practical) benchmark heuristic defense for this problem. An ongoing question is to precisely characterize when and how AT works. The line of work has also proved to be very fruitful for the theoretical community & has produced very general results about problems which can be solved by neural networks, but not other techniques- e.g. kernel methods.

https://arxiv.org/abs/2001.04413

Thanks for the link. It seems like the text is focused on correcting errors across layers. I guess fundamentally there is no difference between the multi-model challenge of correcting errors across models and that of correcting errors across layers. This is dense, but I’m going to dive into the discussion around figure 14 as a starting point.

Thanks again.

Teacher student!!

:) we are all students