Re:Applying machine learning to Infosec
by pilooch on Tuesday July 05, 2016 @01:36PM
(#12038104)
Machine learning is pervasive in the new products targeted at cybersecurity and infosec. From malware to anomaly detection among billions of logs, it's been part of the demand of my customers for at least four years now. Most of the largest companies are already running and testing a series of novel proprietary or custom in-house systems.
The main differences with straight 'supervised' machine learning are the lack of labels, and the unreal volume of data (cheaply produced by machines, in machine time, we're talking microseconds to milliseconds here). So unsupervised learning is king in this domain, and often security operators have to keep an eye on and interpret the results. Another difference with other fields is that datasets are rare, mostly because of privacy as the logs can be very revealing. For this reason the market exhibits a lock that cannot be overcome by everyone I believe. Basically, you sort of need to be in the place already.
Re:Applying machine learning to Infosec
by sp_ on Tuesday July 05, 2016 @02:50PM
(#12038657)
Great summary! By nature of my job (eng lead of a major mobile malware detection team) I have a lot of startups pitch their ML solutions to me. A couple of thoughts:
- There are no publicly available data sets for training available. There are a few small ones and a few old ones, but they don't reflect the reality of 2016. Companies that approach me and pitch me solutions to the malware of 2012 are not useful.
- The majority of mobile malware is based on some kind of social engineering. On a code level these are indistinguishable from legitimate applications (the same APIs are used in the same fashion). The only difference is whether app behavior meets user expectations or not. Making this decision automatically seems intractable so far.
- Malware is not really a well-defined term. There is phishing, toll fraud, Trojans, privilege escalation exploits, ... If you generically look for malware, the signals you will look for are going to approach the complete set of APIs made available by your OS. Your results will just be a giant blob where everything is connected. Pick a single malware category and focus on just that at a time. ML signals for priv esc will look very different from those for phishing.
- ML is sexy. Malware analysis is not. Startups seem to hire too many ML people and not enough malware analysis people. I've had startups pitch to me that had literally zero people on staff who knew what mobile malware actually looked like. They just did anomaly detection and then tossed the results over to my team to verify the results. That's not how it works. We're not your QA team. :)
Re:Applying machine learning to Infosec
by pilooch on Tuesday July 05, 2016 @03:18PM
(#12038817)
Hey, just finished a malware ML custom system for one of the largest european corporations, large enough that some malware is targeted at them. Result is 97% accuracy (they did retrain and check on their own held out dataset). More careful analysis is needed (many malware have high entropy 'zones' that may help the classifier find the right category), but overall it does work.
See the Microsoft / Kaggle challenge on classifying malware families, winning solution is > 99% accuracy IIRC.
Re:Applying machine learning to Infosec
by Eridrus on Tuesday July 05, 2016 @04:57PM
(#12039499)
Can you describe a security setting where 97% accuracy is actually useful? Unless the events you're looking at are low volume or you somehow have much more malicious data than everyone else that seems like a recipe for your results being primarily FPs.
Re:Applying machine learning to Infosec
by lmeyerov on Wednesday July 06, 2016 @01:06AM
(#12041314)
For context, a company can easily get ~1B security-related events a day, so even reporting say 0.1% of those wrong a day means some poor junior analyst has 1,000,000 tickets to slog through. If you expand that to full packet captures as suggested in the article... ouch.
(We do some cool visual analytics work here, including unsupervised learning / classification, and target more of the problem of "given an incident you're already investigating, what else should you now look at from across all your tools?")
Re:Applying machine learning to Infosec
by Cybiote on Tuesday July 05, 2016 @04:17PM
(#12039237)
The 99% means little when it suffers from a similar sort of problem that the immune system has with cancer. Adversary's lack of stationarity vs a fixed model.
Re:Applying machine learning to Infosec
by nickpsecurity on Tuesday July 05, 2016 @06:11PM
(#12039889)
That's what the research under the banner security via diversity and "moving target" are doing. I recall the Hydra firewall from Sentinel did that sort of thing. OpenBSD and grsecurity do in OSS for parts of their OS. Such methods can be combined with these.
Interesting name. Reminds me of a security scheme, Symbiotes, I briefly evaluated on Schneier's blog. Injected security into legacy, embedded applications with various tradeoffs. Where did you get the name from?
Re:Applying machine learning to Infosec
by Fede_V on Wednesday July 06, 2016 @01:08AM
(#12041317)
Focusing on your last point: this is true in a lot of fields.
ML is amazingly powerful, but if you don't have sufficient domain knowledge, or you aren't collaborating very closely with actual experts, you can make very dangerous mistakes. Domain knowledge helps a lot - not just in malware, but in biology, image analysis, etc..
Re:Applying machine learning to Infosec
by kriro on Wednesday July 06, 2016 @09:43AM
(#12042990)
Yeah my initial reaction was...wait a second ML is used in info sec. The most obvious examples are intrusion detection and general network analysis. A less glamorous example is pretty much every spam filter (which also filters fishing stuff and is thus very much infosec-relevant).
I think false positives are a huge problem in the info sec context and less so in other domains. To give a made up example let's say you have ML-based intrusion detection that is hotwired to some sysadmin pager. Even if your false positive rate is fairly low (at acceptable levels in other domains) with the sheer amount of data a bunch of sysadmin calls will be triggered in sum. That is a huge human factor risk as it most likely will result in teaching admins to ignore certain alerts....which of course opens up the interesting attack vector of understanding/"reversing" the ML and triggering false positives a bunch of times until you want to attack through a channel that will trigger a similar response.
Regarding the available data sets: I suppose honeypots/-nets could be created for the purpose of gathering some data sets.
Re:Applying machine learning to Infosec
by pilooch on Tuesday July 05, 2016 @02:13PM
(#12038337)
AFAIK not really. Detecting C&C is not that easy in unsupervised settings. The MIT paper (link in the TC article) makes claims, but this is mostly PR.
There are some reports from Google as well that you may find around, and at least at the time (few years back) they make the same observation we do with my coworkers: the rate of false positive remains high, and thus it is the job of trained security operators to spot the sophisticated attacks.
Empowering the operators with more data mining tools is a good step not yet fulfilled I believe.
Re:Applying machine learning to Infosec
by BrainInAJar on Tuesday July 05, 2016 @04:04PM
(#12039130)
Unfortunately the FP rate is just unacceptably high, you end up overloading your SOC (which means your analysts end up flagging a lot of real threats as FP)
Re:Applying machine learning to Infosec
by drdeca on Tuesday July 05, 2016 @08:02PM
(#12040374)
Well, in order to store the state of the simulation, it would need to store the amount of information needed to describe the state of the simulation. But the information needed to describe the state of the machine doing the simulation, by being enough to describe the state of the machine storing the information about the simulation (including the part of the state which stores the information describing the simulation) , is therefore enough to store the information describing the state of the simulation.
So, therefore, the smallest possible complete description of the state of the simulation is not larger than a complete description of the machine doing the simulating.
So if that is what you mean by more complex / what you mean by more information, then, no, the less-information thing cannot completely simulate the more-information thing.
Re:Applying machine learning to Infosec
by MichaelBurge on Wednesday July 06, 2016 @02:50AM
(#12041574)
It could be that our universe's state can be described by the initial input '1234567', 1 page of differential equations, and an amount of time T to simulate.
An inner universe's state could have an initial input of some string describing all of human history up to T in the outer universe, the same differential equations, and the same amount of time T to simulate.
One might say that the inner universe contains more information, because the smallest unambiguous description of it is larger than the smallest description of its parent universe. However, the outer universe is able to simulate the inner universe.
Re:Applying machine learning to Infosec
by drdeca on Wednesday July 06, 2016 @04:05AM
(#12041790)
But you can specify the information of state of the simulation by specifying 1234567, the differential equations, T, and something to identify the machine that you mean, and that you want the information it stores. This gives you the same info, and would presumably be shorter than "some string describing all of human history up to T" . It would still describe all the info though.
Re:Applying machine learning to Infosec
by MichaelBurge on Wednesday July 06, 2016 @04:34AM
(#12041893)
The purpose of 'Some string describing all of human history' is to have something irreducibly dependent on the first machine. It's extremely unlikely that you'd be able to find an alternate way to describe it that didn't involve describing the outer machine in some way.
You could mark a time and an algorithm for accumulating information, but you now have 2 more pieces of information in the inner machine than the outer one. So the outer machine has somehow managed to simulate the inner machine even though it has less information.
Although... the moment you actually carry out this experiment, it falls apart: Instead of using T, you can just look at time T + U, where U is the amount of time needed to simulate the inner machine. This is because we're sitting here talking about the experiment and eventually carrying it out, so those initial conditions encode the conclusion of the experiment.
This is subtle enough I'd have to look at the math. I can't think through it.
Re:Applying machine learning to Infosec
by duaneb on Tuesday July 05, 2016 @04:16PM
(#12039228)
We already do quantum simulations at tiny levels and at a fraction the pace of real time—so, yes, assuming we can quantify the rules of the simulated system.
Re:Applying machine learning to Infosec
by astazangasta on Tuesday July 05, 2016 @12:54PM
(#12037789)
As machines (computers) grow more sophisticated and start to resemble living things in their complexity, we're going to have to start dramatically improving their regenerative capacity and their ability to self-diagnose. Else the future will be littered with sad, broken robots.
Training a machine implies some sort of evolutionary model (a training set describes a fitness landscape). Maybe this will work (doubtful, across such a large and variable surface), but how about thinking about this on a fundamental design level? How does a computer know it is working properly?
Re:Applying machine learning to Infosec
by digi_owl on Tuesday July 05, 2016 @02:00PM
(#12038256)
> How does a computer know it is working properly?
That is the crux of the issue. Any command that potential malware may give, may also be given legitimately. How one tell those two apart is context. And context is a hard subject even for humans.
Even biology can't get it straight. After all, some of our most resilient diseases exploit the normal signals of cells for their own purposes.
Re:Applying machine learning to Infosec
by ThePhysicist on Tuesday July 05, 2016 @02:37PM
(#12038543)
Capturing data flows and applying machine learning to them seems to be one of the hottest topics in the infosec community right now, so I would not agree that hardly anyone is doing this.
Re:Applying machine learning to Infosec
by ntoshev on Tuesday July 05, 2016 @03:07PM
(#12038752)
It's not just security - I'd expect cache eviction algorithms, schedulers, constraint solvers and many others to also benefit from ML. Anything dealing with hairy problems.
The main differences with straight 'supervised' machine learning are the lack of labels, and the unreal volume of data (cheaply produced by machines, in machine time, we're talking microseconds to milliseconds here). So unsupervised learning is king in this domain, and often security operators have to keep an eye on and interpret the results. Another difference with other fields is that datasets are rare, mostly because of privacy as the logs can be very revealing. For this reason the market exhibits a lock that cannot be overcome by everyone I believe. Basically, you sort of need to be in the place already.
Here is a recent very high level survey on the topic (a TC report, not that bad for once), https://techcrunch.com/2016/07/01/exploiting-machine-learnin...
Here are two of our own Open Source tooling we use the most in application:
- very efficient C++ map-reduce feature generation for logs (for ML and analytics): https://github.com/soprasteria/cybersecurity-miw
- machine learning / deep learning server: https://github.com/beniz/deepdetect
The ML cybersecurity + infosec field is still young, but moving very fast, a lot of new startups and (somewhat opaque) products.
- There are no publicly available data sets for training available. There are a few small ones and a few old ones, but they don't reflect the reality of 2016. Companies that approach me and pitch me solutions to the malware of 2012 are not useful.
- The majority of mobile malware is based on some kind of social engineering. On a code level these are indistinguishable from legitimate applications (the same APIs are used in the same fashion). The only difference is whether app behavior meets user expectations or not. Making this decision automatically seems intractable so far.
- Malware is not really a well-defined term. There is phishing, toll fraud, Trojans, privilege escalation exploits, ... If you generically look for malware, the signals you will look for are going to approach the complete set of APIs made available by your OS. Your results will just be a giant blob where everything is connected. Pick a single malware category and focus on just that at a time. ML signals for priv esc will look very different from those for phishing.
- ML is sexy. Malware analysis is not. Startups seem to hire too many ML people and not enough malware analysis people. I've had startups pitch to me that had literally zero people on staff who knew what mobile malware actually looked like. They just did anomaly detection and then tossed the results over to my team to verify the results. That's not how it works. We're not your QA team. :)
See the Microsoft / Kaggle challenge on classifying malware families, winning solution is > 99% accuracy IIRC.
(We do some cool visual analytics work here, including unsupervised learning / classification, and target more of the problem of "given an incident you're already investigating, what else should you now look at from across all your tools?")
Interesting name. Reminds me of a security scheme, Symbiotes, I briefly evaluated on Schneier's blog. Injected security into legacy, embedded applications with various tradeoffs. Where did you get the name from?
ML is amazingly powerful, but if you don't have sufficient domain knowledge, or you aren't collaborating very closely with actual experts, you can make very dangerous mistakes. Domain knowledge helps a lot - not just in malware, but in biology, image analysis, etc..
I think false positives are a huge problem in the info sec context and less so in other domains. To give a made up example let's say you have ML-based intrusion detection that is hotwired to some sysadmin pager. Even if your false positive rate is fairly low (at acceptable levels in other domains) with the sheer amount of data a bunch of sysadmin calls will be triggered in sum. That is a huge human factor risk as it most likely will result in teaching admins to ignore certain alerts....which of course opens up the interesting attack vector of understanding/"reversing" the ML and triggering false positives a bunch of times until you want to attack through a channel that will trigger a similar response.
Regarding the available data sets: I suppose honeypots/-nets could be created for the purpose of gathering some data sets.
There are some reports from Google as well that you may find around, and at least at the time (few years back) they make the same observation we do with my coworkers: the rate of false positive remains high, and thus it is the job of trained security operators to spot the sophisticated attacks.
Empowering the operators with more data mining tools is a good step not yet fulfilled I believe.
Unrelated, but that gave me a thought to ponder: Will it ever be possible to use computers made inside this universe to model a MORE complex universe?
Has that already happened?
A Turing machine can simulate a python interpreter.
Have you seen the xkcd with the rocks in the desert? https://xkcd.com/505/
So, therefore, the smallest possible complete description of the state of the simulation is not larger than a complete description of the machine doing the simulating.
So if that is what you mean by more complex / what you mean by more information, then, no, the less-information thing cannot completely simulate the more-information thing.
An inner universe's state could have an initial input of some string describing all of human history up to T in the outer universe, the same differential equations, and the same amount of time T to simulate.
One might say that the inner universe contains more information, because the smallest unambiguous description of it is larger than the smallest description of its parent universe. However, the outer universe is able to simulate the inner universe.
You could mark a time and an algorithm for accumulating information, but you now have 2 more pieces of information in the inner machine than the outer one. So the outer machine has somehow managed to simulate the inner machine even though it has less information.
Although... the moment you actually carry out this experiment, it falls apart: Instead of using T, you can just look at time T + U, where U is the amount of time needed to simulate the inner machine. This is because we're sitting here talking about the experiment and eventually carrying it out, so those initial conditions encode the conclusion of the experiment.
This is subtle enough I'd have to look at the math. I can't think through it.
If yes, then can we add something to that simulation, like an extra type of particle?
Training a machine implies some sort of evolutionary model (a training set describes a fitness landscape). Maybe this will work (doubtful, across such a large and variable surface), but how about thinking about this on a fundamental design level? How does a computer know it is working properly?
That is the crux of the issue. Any command that potential malware may give, may also be given legitimately. How one tell those two apart is context. And context is a hard subject even for humans.
Even biology can't get it straight. After all, some of our most resilient diseases exploit the normal signals of cells for their own purposes.
Remember it's not infallible and should not be treated as such.