The infosec community at large is well aware of how unreliable just using md5 checksums to identify malware is. If anything it is the absolute first line of defense for identifying malware, in that it is easy to implement quickly and has a decent enough chance of filtering out low hanging fruit. The biggest use for the checksums between malware researchers is for identifying if they have the same strain of malware as someone else. Identification is mostly not based on checksums, but rather things like YARA rules where different identifying factors of malware are outlined to be compared against binaries. This isn't foolproof either, but there is a rather large ecosystem of malware researchers out there constantly taking samples and releasing rules. I follow a lot of these folks on Twitter and the majority of what they post are their findings on the bajillionth strain of whatever malware is in vogue at the moment. This sort of stuff is going to catch the majority of what will be coming at most people and anything that slips by the first lines of detection usually gets picked up somewhere along the way and passed on to researchers who do an exceptional job of reversing and identifying new malware or strains of old ones. But of course the reliability of that whole ecosystem depends on sensible organization security policy to start with.

In short, md5 sums and signatures are there to protect against the low hanging fruit, spray and pray type malware that's pretty common. If someone wants to target you with uniquely signatured malware they can. Identifying it isn't going to be what stops it, but proper opsec can.

And that's what I don't understand! You say it "has a decent enough chance of filtering", and I believe you -- but this just seems so strange.

It seems to me like it trivial to create a webserver which says "serve the same binary, but put a random ASCII string in bytes 40-48". Or make malware installer which says, "write out the executable file to disk, but put a random value in bytes 80-88". Sure, it won't help against good YARA rule, but it seems really easy to do, and it will frustrate researchers, and even defeat some endpoint protection software, like [0] and [1].

[0] https://help.symantec.com/cs/ATP_3.2/ATP/v106632175_v1273003...

[1] https://docs.mcafee.com/bundle/network-security-platform-9.2...

Its like scam/phishing e-mails with typos in them. Most widescale hacking is lowest effort looking for lowest hanging fruit. If you hack enough servers for your purposes without worrying about checksum randomization then worrying about it is just wasted effort. And you want targets with excessively shitty security postures or else you might actually get tracked down and busted.

Actually not as easy as you’d think to do that, criminals don’t upload malware payloads to their own servers, they use hacked websites (old Wordpress installs, etc) to spread the binaries.

Could they do this? Sure, but it would be a lot more complex than the current method of just finding any target where they can upload files, and the payloads do change very often anyway (sometimes daily) so there is no real need for them to change dynamically

Lucky for us, most criminals are lazy.

I don't think this comes from them being lazy. I think this comes from them not being aware of (1) the defense; and (2) the mitigation. It's an example of security through obscurity.

Or even if they know about the defense and the mitigation, it is additional work. In my work in the formal economy I rarely get to ship the technically best and most complete solution but instead a compromise 'MVP' that'll receive more work only if the problem proves to demand it. I expect the same holds true in the informal economy.

I think, like with many things, basic steps are not taken, through laziness, carelessness or ignorance.

> I follow a lot of these folks on Twitter and the majority of what they post are their findings on the bajillionth strain of whatever malware is in vogue at the moment.

Anyone in particular you recommend following?

krebsonsecurity, notdan, donk_enby are all good cybsec follows. you can probably find others from people that they follow/rt

@malwaremustd1e @malwrhunterteam @0xrb @capesandbox @malware_traffic

The "create customized binaries" issue can be bypassed by treating unknown hashes as suspect and blocking them.

Many systems already do something similar, e.g. I think I've seen Chrome put speed bumps and scary warnings on rare files.

This forces malware authors to choose between the "hey this is the first time we see this file on the planet, it's probably bad" warning and serving the same hash to many victims.

A more extreme approach is binary whitelisting - if the hash isn't explicitly approved, it doesn't run at all.

Hashes are also useful to uniquely identify what you're talking about. Even for a fully polymorphic sample, being able to tell other researchers/defenders "hey, I found ab512f... on my network, and it does X" is useful even if nobody else has the same hash, because then they can get a copy of that sample and analyze it themselves.

Binaries... by binaries you mean pdfs and zip files? Because these are probably the most probable vector of attack rather than exes.

In the end, most malware runs an executable. If it's a ZIP file, it probably contains a binary. If it's a PDF, yes, it could contain an exploit, but nowadays, it's much more likely that it contains a link for the user to click and download malware from. And if it does contain an exploit, there's a decent chance that the exploit will be used to download and install an executable.

It feels like most malware is going backwards from the elaborate schemes of the past to "hey user, please run my malware".

Of course, state sponsored malware _can_ be the exception, although it often uses the same patterns, because they apparently work well enough against most targets.

I don’t think you got the idea of what I am saying.

You can’t limit people to opening only PDFs which you approved. Therefore if a PDF reader has an exploit (be it ability to write to it’s code locations via PDF payload, a scripting language embedded writhin PDF), then you are pwned.

EXE files are far from the only files that allow you to embed code today. Heck, you can embed JS in an SVG image and if you’re unlucky to display that with some not-so-smart image previewed then you’re done.

Fingerprinting is useless for that kind of stuff because it takes away the only good use of computers: sharing of new, unique information.

As I said, exploits are out of scope for binary (i.e. executable) whitelisting and a malicious document could be opened even with binary whitelisting in place and would bypass it - but they're rare in practice.

It raises the bar for the attacker from "trick the user into running your malware" to "have a zero-day for your PDF reader + an escape for whatever sandbox the reader may be put into + fileless malware to actually do something useful after they gained initial execution on the target machine".

Again, not claiming it's unbreakable, but it stops the vast majority of attacks (allowing you to spend time defending against the rest instead of reimaging machines infected with commodity malware).

> How on earth can this be effective?

It's not.

> Did malware authors just forgot about all those and went back to trivial “everyone gets the same image” strategy?

No. But if you make your thing as polymorphic as possible and deliver that automated, i take 1000 samples and look what doesn't change to make a signature.

Also these things exist as webservices these days and criminals pay per use to generate fresh undetected executables.

Just makes more sense to only release a new binary once it had been detected, even if it's just a button click and could be easily automated.

>I take 1000 samples and look what doesn't change to make a signature.

There's fully metamorphic viruses where even the code mutating part is changed. Algorithmically it's equivalent but there's no "static" portion of the virus.

> There's fully metamorphic viruses where even the code mutating part is changed. Algorithmically it's equivalent but there's no "static" portion of the virus.

Never seen it play out that way. There's always something in practice. See you just talked about randomizing the code. But a windows PE isn't just code. It also has an icon. And yes i've seen AVs detect specific icons.

In theory it's easy to not have anything static, in practice, that's the cat and mouse game you're playing.

Look into Mistfall. It would infect other executables by splitting them into basic blocks that end with some branching instruction and inserting metamorphic portions of itself in between blocks and fixing up all of the modified addresses. There literally isn't any "static" portion. Here's a mirror of the author's site, he is Russian but there's a ton of English translations and articles that are a great read.

http://z0mbie.daemonlab.org/

> But a windows PE isn't just code. It also has an icon. And yes i've seen AVs detect specific icons.

And that's how you get overzealous AVs marking executables because they happen to have PE headers...

Why not both? Trivial change (like embedded timestamp in a file) for every user, and advanced polymorphism for the times when the malware is detected.

I bet most researchers have tools to bin files together automatically based on the same MD5. Why make their life easier when this is so easy to defeat?

Because many AV systems assume that any unknown file (unknown to their cloud service) is probably doing this, and thus probably malware.

Note that on macOS many things must be signed and thus modifying them requires a new cryptographic hash. If you’re trying to get through something like the notary service you may not want to submit a bunch of sketchy requests for malware with tweaked hashes.

What they really need is a whitelist of checksums instead. Preferably not md5 though.

That is what Windows Defender Application Control does. It's probably the most cutting edge solution on the market actually.

It's a pure whitelisting solution where every single executable and kernel driver needs to have an approved digital signature or matching hash value or they won't be permitted to run.

It's virtualization assisted and can't be disabled without rebooting and if you use a digitally signed policy and someone tries to remove it, the machine will refuse to boot.

The coolest thing is, it even expands to the scripting languages built-in to Windows so PowerShell execution is restricted according to policy etc.

In practice of course, it's a big pain in the ass to manage - many software are not digitally signed etc.

Every single artifact of every program needs to be digitally signed or have a matching hash in the policy or they won't be permitted to run.

For example, suppose a software installer: the .msi itself is digitally signed so can easily be permitted to run... But then, during installation it unpacks an .exe into %temp% that isn't digitally signed and attempts to run that - oops, won't run. I've come across even Microsoft software that does this.

https://docs.microsoft.com/en-us/windows/security/threat-pro...

How does it handle scripts? E.g. my virus is a python script bundled with the known and clean python executable

It doesn't. Python would need to add support for it.

The built-in scripting languages do, PowerShell enters constrained language mode and IIRC something also happens to VBScript/JScript, haven't even looked at those.

A negative result means little, as you described.

A positive result means an immediate call to action, and a known type of malware. Chances that malware authors would produce a different malware with the same MD5 is close to zero (though, if pulled, it would be quite a practical joke).

That is simply a misuse of checksums. Checksums are for verifying that your binary matches the one you thought you were getting. The more people using them for identifying viruses, the more common polymorphic viruses will be, especially considering how trivial they are to implement.

I remember editing a virus to avoid detection, all I needed to do was switch 2 lines of assembly with each other (the order of execution for these 2 lines didn't matter)

The reality is, if you aren’t stupid, you can actually rob a bank without getting caught, it happens all the time.