APPENDIX 4: INCIDENT MANAGEMENT

Concepts of incident management

Incident management is the practice of preparing for, preventing, detecting, reacting to and recovering from adverse events when they impact business operations, and then documenting lessons to be learned for the future.

The lessons learned from an incident may be valuable in protecting the organization from future incidents and enabling it to respond faster and more accurately, and minimizing the impact of the crisis. However, we see that many organizations do not learn from incidents; instead, they seem to rather hide or move on – avoiding the pain of examining what the conditions were that led up to the crisis. This approach may cause the organization to remain vulnerable to the same problem again and, in fact, it may lead to a much more serious crisis occurring the next time.

The following is not an in-depth review of the entire incident management process, but it does outline some of the key steps for a penetration testing team to take to be prepared for an incident that occurs during their testing.

Incident preparation

Being prepared for an incident is facing the reality that sometimes, despite the best efforts of the organization, things can go wrong. Preparing starts with determining who has the leadership role in a crisis (who “owns” the incident and has the authority to manage the response by allocating resources, incurring costs, and reporting on the crisis). The leader must be provided with the skilled resources – tools and personnel – needed to manage a crisis, should one occur. The personnel must be trained according to the skills they may require.

In the case of pen testing, the testing team must be prepared, in case anything goes wrong during the tests, and also know what to do if they encounter a crisis during the test.

A common challenge that arises in nearly every crisis is that of communication. How will critical personnel be reached? How can a rumor be controlled (especially in the world of social media, where anyone can post anything online in a matter of seconds)? And how can it be ensured that management is kept informed of the crisis, and employees, business partners and customers are provided with accurate information?

The secret to resolving most incidents is preparation – the more time spent on preparation, the more manageable an incident will be.

Incident detection

Incident detection is the skill and practice of knowing what is going on – being alerted to an incident, making the initial decisions as to the scope, source and severity of the incident, and initialing the response teams.

The initial stages of dealing with an incident include triage, containment and escalation of the problem. Once an alert has indicated a potential incident, the response team must determine whether the incident is truly a problem, or a just a false alarm. Once the incident has been confirmed, the first step is to stop it from spreading and contain it as effectively as possible. The less damage caused, the less there will be to repair later.

Perhaps the most important feature of this step is the documentation of the incident. No one can be expected to remember the sequence of events and details of an event following the resolution of the incident. For this reason, it is important to begin creating a record of the incident as soon as possible. This will provide the data needed for the subsequent review, learning and analysis of the incident.

Incident response

Depending on the scope, nature and extent of the incident, the appropriate personnel will need to be engaged and begin the process of responding to – and recovering from – the incident. The primary objective of this phase is to get the business back to a normal state of operations as quickly as possible. In the case of a problem caused by the pen test, the team must be able to contact the key personnel of the client organization and have the knowledge and skills needed to rebuild, stabilize and re-initiate system operations.

In some cases, the response will also require the preservation of evidence. Especially if any criminal activity is involved, the seizure of any evidence must be done with regard to the appropriate standards for handling, documenting, protection, and transportation of evidence. The evidence must be collected in a lawful manner and be as accurate and authentic as possible. The evidence must be documented in a “chain of custody” that details the actions taken with regard to the evidence at all times during its life cycle. There must be an unbroken chain of record for who, what, when, where, and how the evidence was handled at all times.

Follow-up

Following every incident, a follow-up meeting should be held with all involved parties to review the incident and seek out any lessons that could be learned from it. Almost every incident is the result of a series of events that happened in a critical order and, if an organization can learn how to avoid any of the events that led up to the incident, they may prevent similar incidents in the future. Lessons learned may indicate the need for more training, better tools, better monitoring, faster responses, or new or enhanced technical or administrative controls.