EvilModel: Hiding Malware Inside of Neural Network Models [pdf]

Re:EvilModel: Hiding Malware Inside of Neural Network Models... by Imnimo on Tuesday July 20, 2021 @01:11PM (#27896416)

Why would I bother hiding the malware in weights that are actually used by the model? Couldn't I just define a big weight tensor of the appropriate size to hide the malware, declare it part of my model, but never actually make use of it? The paper says:

>To ensure more malware can be embedded, the attacker can introduce more neurons.

It seems like it would be extremely foolish to put the malware in neurons that impact the output if I can just make my own neurons off to the side and get a 0% drop in accuracy!

[ Parent | on HN ]

Re:EvilModel: Hiding Malware Inside of Neural Network Models... by admax88q on Tuesday July 20, 2021 @01:44PM (#27896803)

Consider how you would defend this attack.

If there are extra unconnected neurons, it would be trivial to just remove them, or flag them from anti-virus.

[ Parent | on HN ]

Re:EvilModel: Hiding Malware Inside of Neural Network Models... by Imnimo on Tuesday July 20, 2021 @04:58PM (#27899229)

I can trivially connect the payload subnetwork in such a way that it does not impact the final the output. For example, I could gate it behind relus with large negative biases so that the outputs from the payload neurons are always zero.

Besides, if I'm a defender and am suspicious enough to be modifying my network file, I can just set a random subset of the parameters to zero, simulating dropout, or even just add a small amount of noise to the low order bits of every weight. If the defender is aware enough to modify the network, then I don't think the paper's attack is any more robust than my proposed variation.

[ Parent | on HN ]