Have a clear criteria list when selecting a security orchestration vendor

Selecting security orchestration vendor

Security orchestration, automation and response (SOAR) vendors offer SOCs the best solution against the burgeoning problem of having too many security tools but not enough in-house talent to use them effectively. They enable security operations teams to integrate disparate cybersecurity technologies and processes into a more cohesive security ecosystem, in turn allowing these teams to work more efficiently against the growing onslaught of cyber threats.

The growing number of orchestration vendors and solutions in the market makes it important to be clear on the criteria you will use in selecting the security orchestration vendor that’s right for you. In this post, we highlight key factors to consider when choosing the ideal orchestration vendor for your organization.

Intro to Security Orchestration, Automation and Response (SOAR)

According to Gartner, security orchestration, automation and response (SOAR) equate to technologies that enable organizations to collect security data and alerts from different sources. SOAR helps to combine machine-driven and human-led security operations activities in a way that drives better, more efficient incident analysis and triage according to a standardized set of processes and workflows.

Based on the interplay between security orchestration, automation and incident response, it is easy to see why these elements fit together to form a category of solutions. They encompass what ultimately ladders up to equal security operations – the management of people, processes and technology.

Security orchestration vendors seek to empower analysts and improve incident response through a variety of features. Below we cover six core pieces of functionality you should explore when selecting a security orchestration vendor, features to look for and questions to ask.

Security Orchestration Vendor Criterion #1: Integrations

In a 2017 ESG report on security operations challenges, priorities, and strategies, 29% of the respondents identified poor integration of security tools among the top challenges in security operations. That’s where a security orchestration solution can come in handy. The ability to integrate disparate security solutions is a basic characteristic of security orchestration.

However, not all orchestration solutions are created equal. Therefore, it’s crucial to pick a vendor that not only supports as many widely-used security tools (e.g. Tenable, ArcSight, Splunk, Palo Alto Networks, Symantec, Carbon Black, AlienVault, Anomali, McAfee, to name a few) as possible, but also makes integration of all these disparate tools fast and easy.

In addition, the breadth and depth of its integrations must be as expansive as possible. For instance, choose a vendor that enables the integration of multiple SIEMs as well as non-standard alert sources such as emails. And because some security tools are deployed on-premise while others are now cloud-based, your orchestration vendor should also be able to support both environments.

Finally, consider the flexibility of the vendor with regard to integrations. Look for a security orchestration vendor who can add new integrations quickly and, if your internal team has coding capabilities, look for the ability to create and customize your own integrations.

Questions to ask your prospective security orchestration vendor



How many integrations do you currently support and across which categories?
If you don’t already have an integration I require, how quickly can you build one?
Do you provide an IDE so I can create my own integrations?

Security Orchestration Vendor Criterion #2: SOC Workbench

One of the seemingly trivial, but actually time-consuming (and often confusion-inducing) activities in security operations, is having to switch from one console to another. Console switching is unavoidable in security operations, especially because you typically must run different tools and handle different cases at the same time.

Security operations workbench

For example, on one screen you might be isolating hosts; on another, you might be blacklisting executables; on still another, you could be doing correlation and trending, and yet on one more screen, you might be disabling users. With the many things you need to keep an eye on, this list could get longer.

All that can break your focus as you triage, investigate, analyze, or remediate incidents and alerts. Look for a security orchestration vendor with an interface that minimizes the amount of switching required AND bubbles up the most critical cases so your team can improve its focus and prioritization to bring down response and resolution times.

Screen switching not only takes time, but it requires an understanding of and familiarity with multiple UIs, and the skill with which analyst does this likely varies widely throughout the SOC. Explore how a SOAR vendor’s interface can act as a workbench to help unify your security operations team. Consider ease of use and the ability for analysts of all levels to familiarize themselves and become expert users quickly.

Questions to ask your prospective security orchestration vendor


What is the breadth of activity my team can manage through your interface?

How does the platform prioritize and assign cases?
What skill level is the platform’s UI designed for?

What collaboration capabilities are included in the platform?

Security Orchestration Vendor Criterion #3: Alert Grouping and Case Management

On any given day, it’s normal for a SOC to be bombarded with hundreds of thousands of alerts. While advanced log aggregation tools and SIEMs can help bring together all the event data you need in one place, it’s still very challenging (to put it mildly) to extract the true positives and weed out the false negatives.

And that’s not all. Assuming you’re able to filter out the noise and pinpoint the true positives, you still have to correlate all those alerts (using threat intelligence and other data sources) to understand what’s really happening before you can proceed with incident response and remediation.

Where a security orchestration vendor can provide tangible value is in giving your team the ability to work with grouped or clustered alerts. This must go beyond simply filtering out false positives – which most security orchestration vendors do – to actually grouping related alerts into manageable cases.

Contextual Grouping Enables Case Management

Consider how a security orchestration vendor’s solution translates alerts into cases. If each alert becomes its own case to be worked by an analyst, think about the management impact and collaboration required to effectively handle those cases vs. analysts working cases containing multiple related alerts that can be managed, triaged and closed as a single effort.

Questions to ask your prospective security orchestration vendor

Does your platform group related alerts?

What context is used to determine whether alerts are related?
How are cases created from alerts? Does each alert become its own case?

Security Orchestration Vendor Criterion #4: Visual Investigation

While some alerts/cases can be fully automated and closed, most will still require human analysis. It’s important to understand how a security orchestration solution helps to enable analyst-led activity. Does the solution just run the playbook and hope the analyst figures things out or does it also provide insights or guide the analyst toward solving the puzzle?

To fully understand a particular threat, security analysts would normally draw out key pieces of information from the huge pile of raw data they’ve manually collected from alerts, logs, threat intelligence, and other sources. The analysts would then lay the pieces out to get an overview of the situation, build a storyline and perhaps discover relationships between events.

While this investigation technique is effective in visualizing a threat story line, the common practice still relies heavily on manual methods – like laying things out on a whiteboard – that consumes a ton of precious time. A security orchestration vendor’s solution that mirrors an analyst’s visual investigation process in an interactive interface – reinforced with graphs, timelines, flows, and representations of relevant entities – can significantly speed up investigation and response times.

Be sure to get a look at how a vendor’s platform represents not only the threat story line but the relationship between the entities – IPs, users, files – affected. Ensure your team has the ability to quickly identify relationships, timelines and dig deeper into each entity within a single snapshot. Your team should be able to answer who, what, when, where and how via the information provided through a platform’s visual investigation capabilities.

Graphical, Contextual Threat Storyline

Questions to ask your prospective security orchestration vendor

What are your solution’s visual investigation capabilities?
How are relationships between entities represented?

What level of detail is provided about each entity and how?
How would my analysts build the timeline of a security event?

Security Orchestration Vendor Criterion #5: Playbooks

One of the key ingredients of a successful security operations team is having a good set of response processes or playbooks. Playbooks guide members of your security operations teams on what they need to do and which tools need to come into play if a particular threat alert is raised. So, for instance, you may have a playbook for malware alerts, another for phishing alerts, yet another for data exfiltration, and so on.

The beauty of creating and maintaining playbooks via security orchestration and automation platforms is that it forces the documentation and codifying of existing manual processes and allows for the automation of several tasks. But bear in mind that playbook functionality in a security orchestration solution should be more than just putting tools into automated processes.

Get Started with Standard Playbooks

Look for vendors that provide a breadth of features for playbook creation and customization. Some security orchestration vendors include standard playbooks to help teams get started that can be customized to your organization’s needs and desired levels of automation. Consider the amount of coding capabilities required to build and utilize playbooks within a given vendor’s solution relative to the skill set of your existing team. And determine whether the solution enables your team to simulate alerts to test the effectiveness of your playbooks so you can identify where optimization is needed.

Questions to ask your prospective security orchestration vendor

Do you provide built-in playbooks to help my team get started?
How do you enable my team to create new playbooks?

Is there an IDE?

Does your platform support tests and simulations? 

Security Orchestration Vendor Criterion #6: Reporting

Most of the must-have features we’ve discussed so far are speak directly to the day-to-day impact on incident handlers/responders, security engineers and security analysts who are in the thick of the action. But while these people are typically the most hands-on with a security orchestration solution, SOC managers, CISOs, executives, and other key decision makers stand to benefit.

A security orchestration vendor should be able to help managers and executives understand how their SOC is performing to then make informed decisions about everything from processes and tooling to caseloads and staffing. Not only that, because different stakeholders will want to look at different metrics and KPIs depending on their role, your chosen solution should be able to provide the information they need without adding more burden to your analysts.

Explore vendors that support turnkey and automated reporting, customizable dashboards, templates, and other capabilities that can speed up and simplify reporting.

Questions to ask your prospective security orchestration vendor


What are your dashboarding capabilities?

Can I schedule reports to automatically run and be distributed on a set schedule?
Do you provide reporting templates that can easily be customized for various audiences?

There’s no question security orchestration solutions can elevate your SOC’s capabilities, efficiency and effectiveness tremendously. However, you need to exercise due diligence in selecting a security orchestration vendor in order to get maximum value from your investment. At the end of the day, look for a vendor that will streamline your security operations, reduce missed/uninvestigated alerts, speed up response, enable the creation of consistent/predictable processes, allow better transparency of metrics, and increase your SOCs ability to improve over time.