Vulnerability assessments

A vulnerability assessment is performed to identify, evaluate, quantify, and prioritize security weaknesses in an application or system. Additionally, a vulnerability assessment provides remediation steps to mitigate specific vulnerabilities that are identified in the environment.

There are three general types of vulnerability assessments:

• Port scan (not intensive)
• Vulnerability scan (more intensive)
• Penetration test (most intensive)

Generally, automated network-based scanning tools are used to identify vulnerabilities in applications, systems, and network devices in a network. Sometimes, system-based scanning tools are used to examine configuration settings to identify exploitable vulnerabilities. Often, network- and system-based tools are used together to build a more complete picture of vulnerabilities in an environment.

Penetration testing

Penetration testing (pen testing for short) is the most rigorous form of vulnerability assessment. The level of effort required to perform a penetration test is far higher than for a port scan or vulnerability scan. Typically, an organization will employ a penetration test on a target system or environment when it wants to simulate an actual attack by an adversary.

Log reviews

Reviewing your various security logs on a regular basis (daily, ideally) is a critical step in security control testing. Unfortunately, this important task often ranks only slightly higher than “updating documentation” on many administrators’ “to-do” list. Log reviews often happen only after an incident has already occurred. But that’s not the time to discover that your logging is incomplete or insufficient.

Logging requirements (including any regulatory or legal mandates) need to be clearly defined in an organization’s security policy, including:

• What gets logged, such as
  • Events in network devices, such as firewalls, intrusion prevention systems (IPS), web filters, and data loss prevention (DLP) systems
  • Events in server and workstation operating systems
  • Events in subsystems, such as web servers, database management systems, and application gateways
  • Events in applications
• What’s in the logs, such as
  • Date/time of event
  • Source (and destination, if applicable), protocol, and IP addresses
  • Device, System and/or User ID
  • Event ID and category
  • Event details
• When and how often the logs are reviewed.
• The level of logging (how verbose the logs are)
• How and where the logs are transmitted, stored, and protected; for example:
  • Are the logs stored on a centralized log server or on the local system hard drives?
  • Which secure transmission protocol is used to ensure the integrity of the logging data in transit?
  • How are date and timestamps synchronized (such as an NTP server)?
  • Is encryption of the logs required?
  • Who is authorized access to the logs?
  • Which safeguards are in place to protect the integrity of the logs?
  • How is access to the logs logged?
• How long the logs are retained.
• Which events in logs are triggered to generate alerts, and to whom alerts are sent.

Various log management tools, such as security information and event management (SIEM) systems (discussed in Chapter 9), often are used to help with real-time monitoring, parsing, anomaly detection, and generation of alerts to key personnel.

Synthetic transactions

Synthetic transactions are real-time actions or events that automatically execute on monitored objects. For example, a tool may be used to regularly perform a series of scripted steps on an e-commerce website to measure performance, identify impending performance issues, and simulate the user experience. Thus, synthetic transactions can help an organization proactively test, monitor, and ensure availability (refer to the C-I-A triad in Chapter 3) for critical systems and monitor service-level agreement (SLA) guarantees.

Application performance monitoring tools traditionally have produced such metrics as system uptime, correct processing, and transaction latency. While uptime certainly is an important aspect of availability, it is only one component. Increasingly, reachability (which is a more user- or application-centric metric) is becoming the preferred metric for organizations that focus on customer experience. After all, it doesn’t do your customers much good if your web servers are up 99.999 percent of the time, but Internet connections from their region of the world are slow, DNS doesn’t resolve quickly, or web pages take 5 or 6 seconds to load in an online world that measures responsiveness in milliseconds! Hence, other key metrics for applications are correct processing (perhaps expressed as a percentage, which should be pretty close to 100 percent!) and transaction latency (the length of time it takes for specific types of transactions to complete). These metrics help operations personnel spot application problems.

Code review and testing

Code review and testing (sometimes known as peer review) involves systematically examining application source code to identify bugs, mistakes, inefficiencies, and security vulnerabilities in software programs. Online software repositories, such as Mercurial and Git, enable software developers to manage source code in a collaborative development environment. A code review can be accomplished either manually by carefully examining code changes visually, or by using automated code reviewing software (such as IBM AppScan Source, HP Fortify, and CA Veracode). Different types of code review and testing techniques include

• Pair programming. Pair (or peer) programming is a technique commonly used in agile software development and extreme programming (both discussed in Chapter 10), in which two developers work together and alternate between writing and reviewing code, line by line.
• Lightweight code review. Often performed as part of the development process, consisting of informal walkthroughs, e-mail pass-around, tool-assisted, and/or over-the-shoulder (not recommended for the rare introverted or paranoid developer!) reviews.
• Formal inspections. Structured processes, such as the Fagan inspection, used to identify defects in design documents, requirements specifications, test plans and source code, throughout the development process.

Code review and testing can be invaluable in helping to identify software vulnerabilities such as buffer overflows, script injection vulnerabilities, memory leaks, and race conditions (see Chapter 10 to learn more).

Misuse case testing

The opposite of use case testing (in which normal or expected behavior in a system or application is defined and tested), abuse/misuse case testing is the process of performing unintended and malicious actions in a system or application in order to produce abnormal or unexpected behavior, and thereby identify potential vulnerabilities.

After misuse case testing identifies a potential vulnerability, a use case can be developed to define new requirements for eliminating or mitigating similar vulnerabilities in other programs and applications.

A common technique used in misuse case testing is known as fuzzing. Fuzzing involves the use of automated tools that can produce dozens (or hundreds, or even more) of combinations of input strings to be fed to a program’s data input fields in order to elicit unexpected behavior. Fuzzing is used, for example, in an attempt to successfully attack a program using script injection. Script injection is a technique where a program is tricked into executing commands in various languages, mainly JavaScript and SQL. Tools such as HP WebInspect, IBM AppScan, Acunetix, and Burp Suite have built-in fuzzing and script injection tools that are pretty good at identifying script injection vulnerabilities in software applications.

Test coverage analysis

Test (or code) coverage analysis measures the percentage of source code that is tested by a given test (or validation) suite. Basic coverage criteria typically include

• Branch coverage (for example, every branch at a decision point is executed as TRUE or FALSE)
• Condition (or predicate) coverage (for example, each Boolean expression is evaluated to both TRUE and FALSE)
• Function coverage (for example, every function or subroutine is called)
• Statement coverage (for example, every statement is executed at least once)

For example, a security engineer might use a dynamic application security testing tool (DAST), such as AppScan or WebInspect, to test a travel booking program to determine whether the program has any exploitable security defects. Tools such as these are powerful, and they use a variety of methods to “fuzz” input fields in attempts to discover flaws. But the other thing these tools need to do is fill out forms in every conceivable combination, so that all the program’s code will be executed. In this example of a travel booking tool, these combinations would involve every way in which flights, hotels, or cars could be searched, queried, examined, and finally booked. In a complex program, this can be really daunting. Highly systematic analysis would be needed, to make sure that every possibly combination of conditions is tested so that all of a program’s code is tested.

Interface testing

Interface testing focuses on the interface between different systems and components. It ensures that functions (such as data transfer and control between systems or components) perform correctly and as expected. Interface testing also verifies that any execution errors are properly handled and do not expose any potential security vulnerabilities. Examples of interfaces tested include

• Application programming interfaces (APIs)
• Web services
• Transaction processing gateways
• Physical interfaces, such as keypads, keyboard/mouse/display, and device switches and indicators

APIs, web services, and transaction gateways can often be tested with automated tools such as HP WebInspect, IBM AppScan, and Acunetix, which are also used to test the human-input portion of web applications.

Account management

Management must regularly review user and system accounts and related business processes and records to ensure that privileges are provisioned and de-provisioned appropriately and with proper approvals. The types of reviews include

• All user account provisioning was properly requested, reviewed, approved, and executed.
• All internal personnel transfers result in timely termination of access that is no longer needed.
• All personnel terminations result in timely termination of all access.
• All users holding privileged account access still require it, and their administrative actions are logged.
• All user accounts can be traced back to a proper request, review, and approval.
• All unused user accounts are evaluated to see whether they can be deactivated.
• All users’ access privileges are certified regularly as necessary.

Account management processes are discussed in more detail in Chapter 9.

Management review

Management provides resources and strategic direction for all aspects of an organization, including its information security program. As a part of its overall governance, management will need to review key aspects of the security program. There is no single way that this is done; instead, in the style and with the rigor that management reviews other key activities in an organization, management will review the security program. In larger organizations, this review will likely be quite formal, with executive-level reports created periodically for senior management, including key activities, events, and metrics (think eye candy here). In smaller organizations, this review will probably be a lot less formal. In the smallest organizations, as well as organizations lower security maturity levels, there may not be any management review at all. Management review often includes these activities:

• Review of recent security incidents
• Review of security-related spending
• Review (and ratification) of recent policy changes
• Review (and ratification) of risk treatment decisions
• Review (and ratification) of major changes to security-related processes, and the security-related components of other business processes
• Review of operational- and management-level metrics and risk indicators

The internationally recognized standard, ISO/IEC 27001, “Information technology — Security techniques — Information security management systems — Requirements,” requires that an organization’s management determine what activities and elements in the information security program need to be monitored, the methods to be used, and the individuals or teams that will review them.

Key performance and risk indicators

Key performance and risk indicators are meaningful measurements of key activities in an information security program that can be used to help management at every level better understand how well the security program and its components are performing.

This is easier said than done; here are a few reasons why:

• There is no single set of universal metrics that are applicable to every organization.
• There are different ways to measure performance and risk.
• Executives will want key activities measured in specific ways.
• Maturity levels vary from organization to organization.

Organizations will typically develop metrics and key risk indicators (KRIs) around its key security-related activities to ensure that security processes are operating as expected. Metrics help identify improvement areas by alerting management through unexpected trends.

Some of the focus areas for security metrics include the following:

• Vulnerability management: Operational metrics will include numbers of scans performed, numbers of vulnerabilities identified (and by severity), and numbers of patches applied. Key risk indicators will focus on the elapsed time between the public release of a vulnerability and the completion of patching.
• Incident response: Operational metrics will focus on the numbers and categories of incidents, and whether trends suggest new weaknesses in defenses. Key risk indicators will focus on the time required to realize an incident is in progress (known as dwell time) and the time required to contain and resolve the incident.
• Security awareness training: Operational metrics and key risk indicators generally focus on the completion rate over time.
• Logging and monitoring: Operational metrics generally focus on the numbers and types of events that occur. Key risk indicators focus on the proportion of assets whose logs are being monitored, and the elapsed time between the start of an incident and the time when personnel begin to take action.

Key risk indicators are so-called because they are harbingers of information risk in an organization. Although the development of operational metrics is not all that difficult, security managers often struggle with the problem of developing key risk indicators that make sense to executive management. For example, the vulnerability management process involves the use of one or more vulnerability scanning tools and subsequent remediation efforts. Here, some good operational metrics include numbers of scans performed, numbers of vulnerabilities identified, and the time required to remediate identified vulnerabilities. These metrics, however, will make no sense to management, because they’re lacking business context. However, one or more good key risk indicators can be derived from data in the vulnerability management process. For instance, “percentage of servers supporting manufacturing whose critical security defects are not remediated within ten days” is a great key risk indicator. This metric directly helps management understand how well the vulnerability management process is performing in a specific business context. This is also a good leading indicator of the risk of a potential breach (which exploits an unpatched, vulnerable server) that could impact business operations (manufacturing, in this case).

Backup verification data

Organizations need to routinely review and test system and data backups, and recovery procedures, to ensure they are accurate, complete, and readable. Organizations need to regularly test the ability to actually recover data from backup media, to ensure that they can do so in the event of a hardware malfunction or disaster.

On the surface, this seems easy enough. But, as they say, the devil’s in the details. There are several gotchas and considerations including the following:

• Data recovery versus disaster recovery: There are two main reasons for backing up data:

  • Data recovery: When various circumstances require the recovery of data from a past state.
  • Disaster recovery: When an event has resulted in damage to primary processing systems, necessitating recovery of data onto alternate processing systems.

  For data recovery, you want your backup media (in whatever form) logically and physically near your production systems, so that the logistics of data recovery are simple. However, disaster recovery requires backup media to be far away from the primary processing site so that it is not involved in the same natural disaster. These two are at odds with one another; organizations sometimes solve this by creating two sets of backup media: One stays in the primary processing center, while the other is stored at a secure, offsite storage facility.

• Data integrity: For requests to “roll back” data to an earlier date and time, it is vital to know exactly what data needs to be recovered. Database management systems enforce a rule known as referential integrity. This means that a database cannot be recovered to a state where relationships between indexes, tables, and foreign keys would be broken. This issue often comes into play in larger, distributed systems with multiple databases on different servers, sometimes owned by different organizations.
• Version control: For requests to recover data to an earlier state, personnel also need to be mindful of all changes to programs and database design that are dependent on one another. For instance, rolling data back to a point in time last week may also require that associated computer programs be rolled back, if there were changes in the application that involved both code and data changes. Further, rolling back to an earlier point in time could also involve other components such as run-time libraries, subsystems such as Java, and even operating system versions and patches.
• Staging environments: Depending upon the reason for recovering data from a point in time in the past, it may be appropriate to recover data onto a separate environment. For instance, if certain transactions in an e-commerce environment were lost, it may make sense to recover data including the lost transactions onto a test server, so that those transactions can be found. If older data was recovered onto the primary production environment, this would effectively wipe out transactions from that point in time up to the present.

Training and awareness

Organizations need to measure the participation in and effectiveness of security training and awareness programs. This will ensure that individuals at all levels in the organization understand how to respond to new and evolving threats and vulnerabilities. Security awareness training is discussed in Chapter 3.

Disaster recovery and business continuity

Organizations need to periodically review and test their disaster recovery (DR) and business continuity (BC) plans, to determine whether recovery plans are up-to-date and will result in the successful continuation of critical business processes in the event of a disaster. Disaster recovery and business continuity plan development and testing are discussed in Chapters 3 and 9.

Information security continuous monitoring (ISCM) is defined in NIST SP 800-137 as “maintaining ongoing awareness of information security, vulnerabilities, and threats to support organizational risk management decisions.” An ISCM strategy helps the organization to systematically maintain an effective security management program in a dynamic environment.