Are You Regularly Testing Your Incident Response Processes?


Testing incident response process


Surely you remember it well. Your class being gathered and ushered into the centermost room of your school. Or being taken outside and counting off once you reached your designated place.



Schools run tornado and fire drills so everyone knows what to do, when.

Plus, you don’t want to find out your emergency plans don’t work at the moment disaster actually strikes.

So why aren’t we applying this logic more broadly in cybersecurity?


Everything I Need to Know About Incident Response I Learned in Kindergarten

Security operations leaders would do well to take a queue from the emergency drills of their childhood when it comes to their incident response programs and processes.

What makes sense on paper or the whiteboard often doesn’t work as planned when putting into practice.

As the cybersecurity landscape continues to evolve and threat actor sophistication increases, it is even more important that you not only have incident response processes in place but that you ensure they work consistently. And, of course, you should continuously iterate and improve over time.


Incident Response Processes and the School of Hard Knocks

While many organizations go great lengths to set up effective security operations incident response plans, very few proactively test their processes to ascertain how they will work when faced with a real threat. SANS found that only 33% of organizations periodically review and update their incident response processes, while another 25% only review and update their processes after a major incident. Essentially, teams assume their processes work…until they don’t.


incident response process testing


Source: 2017 SANS Incident Response Survey


Another 25% of organizations revisit their incident response processes after a major incident has occurred. They look at what did and didn’t go well and use the learning as an opportunity to continuously improve. Of course this assumes that the existing processes at least worked reasonably well to address the incident at hand and optimizing is all that is needed.

Essentially, most security practitioners are letting experience be the teacher. Which is a valid approach, but we bet most SOC Managers and CISOs might sleep a bit better at night knowing that their incident response programs have at least been put through their paces before a major incident occurs.

So – how can you put your processes to the test? Let’s take a look at the three main methods used by security operations teams.


How to Test Your Incident Response Processes

Paper Tests

Paper tests are mostly theoretical and may be a first step for security operations teams who don’t have well-documented incident response processes. That said, these kinds of tests still leave too much room for error and should only be used to look for small process changes, updates in contact info or other small changes that have occurred since the last time the process was tested.


incident response process tabletop exercise


Tabletop Exercises

Tabletop exercises are just that – stakeholders around a table running through a security event scenario. This technique allows teams to review and practice the various actions detailed in an incident response process. Although tabletop exercises can appear informal, the best ones are extremely focused and include members from across the organization.

To make these exercises effective, you should prepare well in advance, ensure the right stakeholders participate and make the scenario as real as possible. Allow for up to half a day to really put key processes through their paces and troubleshoot as you go.


Simulated Attacks

A fully simulated attack is the most effective way of pressure testing your incident response processes as it uses real life, controlled attacks to see how an organization will respond when hit by an external threat. For instance, an organization can simulate the deployment of a known threat

and run the associated incident response plan to test its efficacy.

A simulated attack is typically the most resource-intensive type of test, often involving all of the stakeholders that would be part of your incident response war room, such as public relations and legal. Simulated attacks not only test what your team will do, but how they will do it.

Simulated attacks are often still done tabletop style, but an increasing number of security orchestration tools give your team the ability to simulate attacks and run your playbooks. This method can give your team the clearest picture – a full, real-time look at how your incident response process will perform in the event of an actual attack.




Optimizing Your Incident Response Processes

Testing your incident response processes yield two important results – a clear understanding of whether your plan is likely to work and a list of gaps that should be addressed. There’s absolutely no point running simulated attacks if they’ll play no role in optimizing the incident response processes. Lessons learned during these exercises must be properly documented for them to have real, lasting value for your security operations team.

Your incident response playbooks should always be updated after they’ve been put to use, whether in a simulated scenario or as part of real security incident triage and remediation. And, through testing, you should identify opportunities to apply automation to your incident response processes to expedite remediation and keep your analysts focused on the highest value tasks.


The Role of Playbooks

Everything we’ve discussed in this article assumes your organization has some level of documentation for your incident response processes. If you find yourself in the position of working from tribal knowledge or paper-documented processes, the first step is to codify each of your processes into playbooks.

Playbooks ensure that everyone in your organization is on the same page, will execute processes the same way and knows what their role is in the event of an incident. As with attack simulations, there are a variety of ways to approach playbook creation, including automated playbooks available within many security orchestration platforms.

Once you’ve detailed the various steps, roles and entities involved in responding to a particular type of attack, you can then test your playbook either through a tabletop exercise or full attack simulation. And from there, you can determine opportunities for optimization, including the application of security automation.




Having the best incident response plan is only as good the paper it’s written on if it fails to provide a suitable response to a threat. Your incident response processes should be codified, documented and regularly pressure tested for vulnerabilities. And you must ensure that playbooks exist and are regularly updated to reflect lessons learned from the tests and actual incidents.

It’s always better to be prepared for a disaster that never comes than to find you are woefully unprepared when an incident actually occurs.