(The following is a guest post written by Alissa Knight, an ethical hacker, entrepreneur and author.)
The average lifespan of an enterprise security information and event management solution (SIEM) has decayed to approximately 18 to 24 months, according to CMS Distribution, largely propelled, many believe, by a systemic lack of expertise and inadequate staffing.
The high signal-to-noise ratio of traditional SIEM solutions, combined with a systemic lack of staffing , have impelled a new generation of SIEM complemented by security orchestration, automation, and response (SOAR) functionality.
This begs to ask: What is the difference between next-gen (NG) SIEM solutions and SOAR, and how do the new requirements of what defines a NG SIEM compare or contrast to the capabilities of a SOAR platform?
But first, it’s essential we define what capabilities must be met by a SIEM solution to be considered NG.
Behind the Difference
As this paper will outline, the sharp divide between both sides of the aisle on whether NG SIEM is a fair comparison to SOAR solutions arose with the introduction of automation and response features to traditional SIEM platforms. Some SIEM solutions have even begun adding playbooks similar to what can be found in SOAR technology. Yet as I’ll detail in this paper, NG SIEM solutions that have added SOAR features do not replace or offer an alternative to what a full SOAR solution can provide.
History of SIEM
The first true SIEM to appear in the market was likely Intellitactics in 1996. The product category at the time was colloquially referred to as network security management (NSM). Later, the term would be supplanted with the phrase SIEM by the analyst community in 2005 by Mark Nicolett and Amrit Williams of Gartner.
Intellitactics took home the award for being the first-to-market three years before NetForensics, which was then usurped by ArcSight one year later in 2000.
However, these traditional, first-generation SIEM solutions began to quickly prove challenging for already talent-starved organizations which had few internal cybersecurity resources who certainly didn’t have time to sit in front of a SIEM day in and day out tuning, creating content rules or validating false positives, while looking out for false negatives.
The term “event (or alert) fatigue” became a present-day challenge, giving rise to a new market of cybersecurity service providers ushered in by Counterpane, Riptech, and the many other managed security service providers (MSSPs) that followed. These vendors offered to take over the burden of monitoring. MSSPs offered hope to organizations by acting as a triage for level 1 and level 2 event analysis for organizations unable to staff an internal security operations center (SOC).
First-generation SIEM solutions initially started out as log aggregators historically powered by relational databases, capping their ability to provide real-time response as databases grew under the load of more log sources such as ArcSight (which actually had two separate database backends — MySQL and PostgreSQL — storing metadata in the Postgres database about the events and trends while active lists and other information were stored in the MySQL database).
The introduction of correlation engines began to give intelligence to first-generation SIEM in an attempt to address the event fatigue problem caused by false positives and an effort to create the equation (A + B + C is related to the same event and = something bad).
Rise of Next-Gen
Despite the introduction of correlation engines, SIEM still fell short of expectations. SIEM technologies were unable to aggregate and correlate all log and event data from on-prem and cloud workloads, software-as-a-service solutions, and system and network telemetry data, as well as provide the capability to perform automated response for detected threats. Enter the NG SIEM.
In order to qualify as a NG SIEM, the solution needs to leverage NOSQL databases, such as Hadoop, Elastic, Spark and other technologies that simply weren’t available in the early part of the 21st century. Data warehouses that were used by legacy SIEM solutions included MySQL, PostgreSQL, MSSQL, and even Oracle, overwhelming the backend, rendering them unusable over time, and preventing organizations from sending any new raw event data to their SIEM unless it was absolutely necessary.
Over the last two decades, data science has matured at an evolutionary pace, obviating the need for false positive-prone pattern-matching engines, also referred to as signatures. NG SIEM solutions incorporate machine learning (ML) capabilities to leverage supervised and unsupervised models to cluster like events together and identify anomalies from learned behavior. This helps prevent overwhelming the analyst by deafening her with too much noise.
One of the most prevalent themes to become part of the daily narrative in security operations (SecOps) is the concept of applying context to security to determine if an event should be considered a true positive. This is the idea that the SIEM solution should be able to take its understanding of a given asset and apply context to an event affecting that asset, if it indeed is relevant.
For example, an event may trigger from a network detection and response (NDR) solution that an Apache buffer overflow attack was detected that may be real, but the target IP address is running Windows and the IIS web server. Context in this case would therefore not apply, despite it being a real attack, saving an analyst time in having to further investigate.
By incorporating more intelligence into the traditional SIEM, which makes it aware of not just asset information but also the learned behaviors of users in the environment, gives it the capability to apply user entity behavior analytics (UEBA). NG SIEM solutions don’t simply identify an event as being “bad or good;” rather, using ML models, they assign a type of score to an event and when that score exceeds a specified threshold, is then presented to the analyst for further analysis.
Legacy SIEM solutions typically presented events by categorizing them into tables of high, medium, or low severity without much more context beyond the potential severity of the event. In the context of UEBA, a NG SIEM can quickly identify anomalous behavior when, for example, an employee suddenly demonstrates behavior not previously seen by the SIEM (such as VPNing in from Ukraine at 2 a.m. when the individual lives in California and has never VPNed in outside of work hours).
Because legacy SIEM products didn’t have much in the way of asset and infrastructure awareness, they were incapable of identifying lateral movement following a foothold by an adversary. Conversely, NG SIEM solutions are now capable of tracking the lateral movement of adversaries as they pivot from one asset to another in an on-prem network or even cloud workloads.
Just like in the investigation of a crime scene, the primary job of an investigator is to piece together the events against an established timeline. Timeline generation of related events is a hallmark capability of NG SIEM solutions that previously had to be reconstructed manually by analysts in a legacy SIEM.
Arguably the most powerful capability to be added to a NG SIEM is the capability to perform automated responses to known threats that are predefined by incident response playbooks. Unlike their ancestral legacy SIEM solutions, NG SIEMs are capable of not simply pulling event data from applications and systems but also stacking workflow automation on top of orchestration, such as pushing response actions to devices like firewalls or intrusion prevention systems (IPSs) in response to detected threats. This makes NG SIEM similar in capability to SOAR technology, thus creating the confusion in the market.
Finally, NG SIEM solutions have integrated threat hunting capabilities, allowing analysts to uncover suspicious activity and vulnerabilities in their environment, as well as monitor threat intelligence feeds to uncover potential issues, adversaries and indicators of compromise.
SOAR Enters the Arms Race
Before we discuss SOAR, it is important to first introduce the concept of MTTR or mean-time-to-resolution and MTTD mean-time-to-detection. The concept of MTTR first originated in deskside support/IT support and signified the duration of when a problem ticket was first reported and subsequently resolved by a technician. Having since been adopted by cybersecurity analysts, its meaning remains the same with the exception that MTTR is used to define the span of time between when a confirmed cybersecurity incident is first triaged to when it is eventually resolved. MTTD, meanwhile, refers to when an adversary first employs the tactics and techniques used to obtain a foothold on a target network to when she is eventually detected by a network or endpoint security control.
SOAR was conceived to help address the SIEM challenge of event/alert fatigue and the global talent shortage in cybersecurity for organizations to effectively staff a SIEM deployment. SOAR effectively streamlines what were once manual tasks as a way of removing human error from the MTTR/MTTD loop through automation and orchestration, powered by playbooks, while reducing the tedium and overtaxing nature of threat analysis.
In summary, unlike NG SIEM, SOAR is an integration platform that glues an organization’s numerous SecOps tools together and automates them using incident response playbooks that can be executed automatically or with a single click by a SOC analyst. SOAR also facilitates case management with a purpose-built issue tracking system (ITS) for codifying security event analysis and response workflows.
The best way to compare and contrast NG SIEM from SOAR platforms is to consider SIEM solutions to be a system of record and SOAR platforms to be a system of action. This doesn’t remove the need for a SIEM. Instead, when combined with SOAR, a SIEM is more effective in reducing MTTD/MTTR, addresses the challenge of inadequate staffing, and lowers the high signal-to-noise ratio common in many SOCs.
SOAR VIS-A-VIS NG SIEM
As Albert Einstein once said: “Not everything that counts can be counted, and not everything that can be counted counts.”
Many foresaw the collision of the SOAR and NG SIEM worlds as NG SIEM companies began acquiring SOAR companies with the ultimate objective of integrating SOAR capabilities into their SIEM platform or expanding the integration between the two.
Therefore any debate of SOAR versus NG SIEM is a relative one. NG SIEM platforms that integrate SOAR capabilities because of the necessity to support SIEM functions will not incorporate all of the capabilities of a dedicated SOAR platform. By virtue of adding playbooks and automated response to a NG SIEM does not in and of itself make it better or more effective at automated response and orchestration offered by a dedicated SOAR solution.
Therefore the question doesn’t boil down to whether one can be done better than the other, rather, can your security operations program function effectively without all of the capabilities offered by a dedicated SOAR platform versus adopting a NG SIEM solution with a few of the automations offered by SOAR?
Alissa Knight is a 20-year veteran in cybersecurity as a penetration tester and vulnerability researcher. She is also a serial entrepreneur having sold two previous startups in cybersecurity in successful M&A transactions and recently reinvented herself as a full-time writer in her impossible recovery as a former hacker.