Contents
Part I Introducing Security Metrics
Security Vulnerability and Incident Statistics
The Dissatisfying State of Security Metrics: Lessons from Other Industries
Reassessing Our Ideas About Security Metrics
2 Designing Effective Security Metrics
Defining Metrics and Measurement
Nothing Either Good or Bad, but Thinking Makes It So
GQM for Better Security Metrics
Measuring Compliance to a Regulation or Standard
Applying GQM to Your Own Security Measurements
Data Sources for Security Metrics
We Have Metrics and Data—Now What?
Case Study 1: In Search of Enterprise Metrics
Scenario One: Our New Vulnerability Management Program
Scenario Three: The Value of a Slide
Scenario Four: The Monitoring Program
Scenario Five: What Cost, the Truth?
Part II Implementing Security Metrics
4 The Security Process Management Framework
Managing Security as a Business Process
The Security Improvement Program
Getting Buy-in: Where’s the Forest?
5 Analyzing Security Metrics Data
What Do You Want to Accomplish?
Qualitative and Mixed Method Analysis
6 Designing the Security Measurement Project
Phase One: Build a Project Plan and Assemble the Team
Phase Two: Gather the Metrics Data
Storing and Protecting Metrics Data
Phase Three: Analyze the Metrics Data and Build Conclusions
Phase Four: Present the Results
Case Study 2: Normalizing Tool Data in a Security Posture Assessment
Background: Overview of the SPA Service
PART III Exploring Security Measurement Projects
7 Measuring Security Operations
Sample Metrics for Security Operations
Sample Measurement Projects for Security Operations
SMP: Internal Vulnerability Assessment
8 Measuring Compliance and Conformance
The Challenges of Measuring Compliance
Confusion Among Related Standards
Confusion Across Multiple Frameworks
Sample Measurement Projects for Compliance and Conformance
Creating a Rationalized Common Control Framework
Mapping Assessments to Compliance Frameworks
Analyzing the Readability of Security Policy Documents
9 Measuring Security Cost and Value
Sample Measurement Projects for Compliance and Conformance
Measuring the Likelihood of Reported Personally Identifiable Information (PII) Disclosures
Measuring the Cost Benefits of Outsourcing a Security Incident Monitoring Process
Measuring the Cost of Security Processes
The Importance of Data to Measuring Cost and Value
10 Measuring People, Organizations, and Culture
Sample Measurement Projects for People, Organizations, and Culture
Measuring the Security Orientation of Company Stakeholders
An Ethnography of Physical Security Practices
Case Study 3: Web Application Vulnerabilities
Outcomes, Timelines, Resources
Initial Reporting with “Dirty Data”
Determining Which Source to Use
Working with Stakeholders to Perform Data Cleansing
Follow-up with Reports and Discussions with Stakeholders
Lesson Learned: Fix the Process, and Then Automate
Lesson Learned: Don’t Wait for Perfect Data Before Reporting
PART IV Beyond Security Metrics
11 The Security Improvement Program
Moving from Projects to Programs
Managing Security Measurement with a Security Improvement Program
Governance of Security Measurement
The SIP: It’s Still about the Data
Documenting Your Security Measurement Projects
Sharing Your Security Measurement Results
Collaborating Across Projects and Over Time
Security Improvement Is Habit Forming
Case Study: A SIP for Insider Threat Measurement
12 Learning Security: Different Contexts for Security Process Management
Three Learning Styles for IT Security Metrics
Standardized Testing: Measurement in ISO/IEC 27004
The School of Life: Basili’s Experience Factory
Mindfulness: Karl Weick and the High-Reliability Organization
Case Study 4: Getting Management Buy-in for the Security Metrics Program
Corporations vs. Higher Ed: Who’s Crazier?