At $dayjob I have access to the Azure Active Directory "unusual user activity" security alerts in an environment with about 15K staff.
In my experience, the reports are accurate, in the sense that they get triggered as advertised by mis-use of IT resources. However, 95% of the time it's just staff being "naughty" instead of actual hackers.
For example, the "impossible travel" one gets triggered regularly. About 80% of the time it's because someone forgot to turn off the VPN they use to watch foreign Netflix. The other 20% of the time is because they were sharing credentials, which is against every IT security policy ever.
Even just the use of a VPN by itself is red flag. VPN providers are notoriously untrustworthy, many of them teetering on the edge of being outright malware. Certainly they all collect far too much metadata and sell it to the highest bidder. No such VPN product has any legitimate use on a corporate device.
Not to mention that corporate traffic is now looping out of the country into another country and back for no good reason...
You’d like to think so, but...
As I tell people, "We don't care if you browse Reddit, we only care if you start doing things an employee shouldn't".
But to answer your questions we would just ingest those alerts into Splunk, build a KB on how to handle the alerts when they trigger and then begin the process of filtering out the noise. The SOC Analyst who works these alerts will get numb to them but still pick out the unusual ones to investigate.
Aah yes, Splunk....
Good tool, but realistically only viable for those working for a corporate-sized employer with corporate-sized pockets filled with $$$$$$$.
Most people can't afford to simply dump it all into Splunk and (if they are using Splunk at all) have to pre-filter first. Which kind of defeats the point of Splunk if they're already doing the hard work outside, and so might was well use some cheaper (F)OSS tool.
Right now it's 100% free, because I'm just looking for user feedback. I think/hope there is an opening in the market for folks looking for an easy-to-use, but powerful tool like Splunk, but can't afford their hefty price tag. All feedback welcomed!
Also created an open-source ETL tool: https://log-ship.com
For one part of the org, occasionally doing 100x data transfers may not be out of the ordinary, while for others it may be exceptional. But it might be anomalous if its 100x data transfer and hitting services no one on your team has ever hit (indicating perhaps scanning/scraping and exfiltrating data, but you don't need to explicitly specify that as an alert).
So instead of hiring the same constant number of security monitors/auditors independently of the organization size, can't you "solve" this problem by hiring one per every N employees, and having them monitor solely that group?
It sounds to me like either this solution should work (a "large" organization should be able to afford this!), or the problem isn't really related to the size of the organization.
Additionally, security is one of those things it's hard to get C-suite execs to pay attention to and spend money on. When it's working, you don't notice it. And if your 5 person security team has worked fine for the past 5 years as the org has grown, why should you suddenly scale it up to 10? Everything is fine! That is...until it isn't. But rarely do people think ahead like this.
Defense Logistics Agency in 2018 reported 264 applications in use, which was down from 1200 in 2013[1]. DLA only represent about 0.9% of the total DOD workforce[2] and it appears the application reduction could be due to consolidation of applications together, making application counts appear better but not doing much to reduce true complexity.
Given many systems have a single subject matter expert (or sometimes none), what is a cyber security analyst responsible for 100's of applications going to be able to reason about an event raised from a system with a name as vague as "DLA Action Item Tickler Report"? Was Bob OK to send the XY12 report via e-mail to Oscar? Who knows? It's likely no one has asked that type of question before and the only person who knows enough about the system to answer the question is the person who caused the event to be generated.
Some organisations are quite simple (despite having similar finances and employee counts) because they largely consist of simple repeatable processes performed by 1000's of employees using a single software product. Some organisations (particularly the DOD) are very complex because every employee is doing something unique across 100's of different software products.
[1] https://www.dla.mil/About-DLA/News/News-Article-View/Article...
[2] https://en.wikipedia.org/wiki/United_States_Department_of_De...
Like this:
I log into system foo-bar-baz, which one person accesses once or twice a year. Maybe it's christmas setup, maybe it's for daylight savings time, maybe it's for a new release of Ubuntu, whatever. I have all the relevant credentials, I am responsible for foo-bar-baz, and I have an authorised change request.
Three days later, the security team sends me a slack message, asking if I accessed system foo-bar-baz.
I tell them yes.
1.6 Behavioral, Contextual ID and Biometrics & 7.4 User and Entity Behavior Analytics - focus monitoring/auditing on accounts that all of a sudden transfer 10GB data when they usually only transfer only 100MB/day, or where the employee has had to be asked for that one time of the year to login on a weekend at an office they don't usually visit.
5.1 Data Flow Mapping - detect unexpected egress of data by defining ahead of time the volumes of data being transferred between systems (e.g. 2AM backup transfers 100GB to systemX and between 9AM-5PM there is a usual data transfer rate of 1MB/s therefore 100MB/s transfer rate at 1PM would raise an alert).
How well do these techniques work in practice, particularly in a huge organisation? I would have thought the number of false positives would be very high and the people monitoring the anomalous behaviour wouldn't have much or any context to know whether something is legitimate or not.
A more feasible approach may be system owners installing a new system would have to specify rate limits (including per time of day, per API call and/or per user) and would have to lodge as part of a change request whether these limits need to be temporarily increased to cater for a one-off or rare event such as a major system upgrade. But given that some of the other techniques listed indicate a lack of awareness of what software is installed and is in use, it seems unlikely that specification of rate limits would happen any time soon.