News for hackers. Stuff that matters.
Re:EvilModel: Hiding Malware Inside of Neural Network Models... by maffydub on Tuesday July 20, 2021 @08:03AM (#27892712)
I'm not sure why this is specific to malware. Isn't this just steganography? You could equally hide malware in a compressed image.

Maybe the amount of data you can hide is higher, but that's primarily because they're storing all their weights as 32-bit floats which is overkill for inference.

...and I guess the fact you can retrain after hiding your malware to increase your inference accuracy again is maybe interesting?

Re:EvilModel: Hiding Malware Inside of Neural Network Models... by nuclearnice1 on Tuesday July 20, 2021 @08:40AM (#27893009)
> I'm not sure why this is specific to malware. Isn't this just steganography? You could equally hide malware in a compressed image.

Correct.

Paper. 3rd paragraph, page 1: “For delivering large-sized malware, some attackers attach the malware to benign-look carriers, like images, documents, compressed files, etc. [5] The malware is attached to the back of the carrier while keeping the carrier’s structure not damaged. Although they are often invisible to ordinary users, it is easy to detect them by antivirus engines. Another way to hide messages is steganography.”

Re:EvilModel: Hiding Malware Inside of Neural Network Models... by joosters on Tuesday July 20, 2021 @08:10AM (#27892773)
Maybe because of the density of the payload? Hiding 37MB of data inside of 178MB of images would not be possible without severely degrading the image quality - and they are simple to check. Whereas the NN model continued to work with very little quality loss.

(You could easily append 37MB of hidden data to some images - e.g. adding an invisible extra layer to the image, but this paper details a technique where you don't alter the file size)

Re:EvilModel: Hiding Malware Inside of Neural Network Models... by mirker on Tuesday July 20, 2021 @08:20AM (#27892868)
They use AlexNet, which is way out of date at this point and I believe understood to be parameter inefficient. Most of those model parameters are likely useless, and at least less useful than those of other newer models. The model they chose is basically zero-padded and they are appending to the zero-d region.

Also, the newest JPEG standard is nearly Turing complete, so you can possibly compress the data into a program which is the same size as the Kolmogorov complexity of the data.

Re:EvilModel: Hiding Malware Inside of Neural Network Models... by high_byte on Tuesday July 20, 2021 @08:22AM (#27892888)
not true. 178mb bmp could become few mb of jpg, even kb for mspaint quality images (ie. few colors, low frequency)
Re:EvilModel: Hiding Malware Inside of Neural Network Models... by api on Tuesday July 20, 2021 @09:03AM (#27893203)
Neural networks can be Turing-complete, which could in theory allow embedded malware to actually run and do things. I can imagine a compiler that targeted neural networks and allowed programs to be compiled to run silently within them.

What it could do is of course highly dependent on what the neural network is doing and how it's embedded in an application. In many cases it would not be able to do much, but if the neural network is controlling something or has any mechanism to feed back into the app and execute commands...

Then "attacks only get better."

Re:EvilModel: Hiding Malware Inside of Neural Network Models... by xyzzy21 on Tuesday July 20, 2021 @12:48PM (#27896103)
The fact you can't "explain" how NN/ML arrives at its answers easily perhaps makes it a bit different but yeah, mostly. With stenography you overall statistic changes that can sometimes indicate it (though those can be spoofed away).