So you took the proactive step to get a budget and purchase a SOAR, congratulations.

The funny thing about budgets is that if they turn out not to result in value/benefit they can disappear fast. Now whether you used the simple guide to selecting a SOAR (Pragmatic SOAR Selection) we recently published or some other means, you are now entering a critical phase in your SOAR journey. To ensure your SOAR budget remains intact into the future you need to show that the SOAR is delivering value. And, if worse case the value hasn’t materialized you need to be prepared to select a new vendor fast. Here is a simple process you can follow to ensure you know how well, or poorly your SOAR is performing.

Establishing a Baseline

With security products, it is always a challenge to understand how much value they are bringing to the team and company. To that end, before installing one piece of software or running one automation take a snapshot of the existing SOC analytics. These analytics will be critical to keep on hand when renewal time comes so that it is easy to show value and/or hold the vendor accountable for performance.

It’s recommended to capture stats such as:

  • Number of alerts/month
  • Number of false positives/month
  • Investigations per analysts/month
  • Mean-time to resolution/month

Fortunately, the leading SOAR providers track these types of stats automatically so no extraordinary steps need to be taken to pull these stats after the SOAR is implemented.

Planning SOAR Rollout

Best practice for any rollout is to “measure twice, cut one”. While excited about this new investigation platform, opening up the platform to all analysts immediately can actually cause frustration. Take a pragmatic approach to rollout, such as described below.

Phase I Rollout

The analysts that were part of the evaluation team should be the first adopters of the new tool. They should work closely with the security engineer to ensure all integrations required to complete investigations are in place in advance of the first alerts being fed into the system. They should also, working with the vendor, develop the onboarding plan for the rest of the SOC team. This plan should include initial training, generally available from the vendor, as well as a plan for transitioning all existing runbooks into the SOAR.

Once a minimum viable solution has been architected the analysts can begin working through initial investigations. Working closely with the vendor’s services team during this phase is essential as there will undoubtedly be several customizations required to the playbooks as the analysts begin working live cases.

Once all these tweaks have been worked out the SOC manager can begin rolling out the platform to the larger team.

Phase II Rollout

Phase II will vary depending on the size of the team and the number of playbooks either transitioned into the platform or identified as needing to be created. At some point, however, the vendor services team will begin to phase out of the project which means the analysts who participated in phase I need to be prepared to take ownership of the project.

This, of course, does not mean the vendor no longer provides support.

Any reputable SOAR providers will assuredly provide customer support via telephone, email, chat, and offer a customer portal. The analysts who now own the project should ensure they have access to this customer portal so that they can gain insights from existing customers and have ready access to the vendor support team.


After several months on the new platform, it will be important to pull a set of metrics to ensure the solution is delivering as expected. Here is where the baseline that was created prior to deployment is critical. As mentioned leading SOAR vendors provide these real-time metrics so comparing to the baseline should be easy. If the SOAR is performing as expected no further changes may be needed, however, if the SOAR is not delivering improved efficiency and analysts performance is not improving it may necessitate a discussion with the vendor. The point here is that it is better to identify an issue, or worst case, a wrong tool selection sooner than later. In many cases, some minor changes may all that is required for the tool to begin delivering the expected results.

Transitioning to Steady State

With the SOAR in place, the playbooks delivering the automation required, and the SOC performance being tracked the SOC has now undergone a significant transition. Most organizations who successfully move to a SOAR-based SOC find analyst effectiveness is not only improved but their overall satisfaction with their job increases as well. This secondary benefit can help minimize SOC analyst attrition, making the SOC stronger and its ability to provide superior security for the organization improved.

Further, some organizations, now with never-before-seen metrics on their security stack, can look at eliminating redundant or poor performing security controls. This benefits the entire team as that budget previously dedicated to those other controls can now be deployed for more resources, better hardware, or the like.

The Road Ahead

Keeping your SOC efficient and effective isn’t a point in time goal. As threats evolve and the skills in your SOC change its important to make sure the technology you trust to run your SOC is evolving with your needs. If your SOAR begins to show signs of lagging hold your vendor accountable. If they are not responsive to your needs, take charge and make a change. Once you have had a SOAR solution deployed you have done the heavy lifting already so moving to a new solution, while not plug and play, will certainly be easier than the first time around.