Re:Planting Undetectable Backdoors in Machine Learning Models
by moyix on Saturday February 25, 2023 @06:03PM
(#34941348)
Adversarial attacks are inference-time, backdoors are training time. This paper isn't the first to propose the idea of backdooring DNNs (I believe our paper [1], concurrently with a couple others [2,3], did that). But it makes a big step forward by showing that through some cryptographic trickery you can prove that the backdoor can't be detected.
[1] https://arxiv.org/abs/1708.06733
[2] https://www.ndss-symposium.org/wp-content/uploads/2018/02/nd...
Re:Planting Undetectable Backdoors in Machine Learning Models
by version_five on Saturday February 25, 2023 @06:11PM
(#34941395)
Isn't the backdoor essentially equivalent to just simplifying an inference time adversarial attack?
Re:Planting Undetectable Backdoors in Machine Learning Models
by moyix on Saturday February 25, 2023 @10:24PM
(#34942759)
Even if we found a solution to inference-time adversarial attacks tomorrow, backdoor attacks would still be possible, which makes them pretty different IMO.
Re:Planting Undetectable Backdoors in Machine Learning Models
by cryptohell on Sunday February 26, 2023 @12:24AM
(#34943289)
There are several differences:
1. Empirically, networks have many adversarial examples. It doesn't mean though that there are adversarial examples everywhere. They show that any point can be slightly changed to get whichever output.
2. Some training algorithms that already exist or will exist are meant to be robust. They show that even with a robust algorithm the backdoor will still exist.
3. As you said, they show that finding the backdoored point is also efficient to the key holder.
Re:Planting Undetectable Backdoors in Machine Learning Models
by ShredKazoo on Sunday February 26, 2023 @04:31PM
(#34949690)
>through some cryptographic trickery you can prove that the backdoor can't be detected.
Can you explain more about this? E.g. in the worst case, if I know the learning algorithm, I could retrain the model myself and notice the difference, right? What is the threat model exactly?
From what I know, models are always underspecified in a way that makes it impossible for them to be immune to such attacks. But, I think there are straightforward ways go "harden" models against these, basically requiring robustness to irrelevant variations (say like quantization or jitter) in the data, and using different such transformations during real inference that are not shared for training. (Or some variation of this).
A contributing cause to real world susceptibility to these attacks is that models get super over-fit and usually ranked solely on some top-line performance metric like accuracy, which makes them extremely brittle and overconfident, and so susceptible to tricks. Ironically a slightly crappier model may be much more immune to this