Index
As this ebook edition doesn't have fixed pagination, the page numbers below are hyperlinked for reference only, based on the printed edition of this book.
A
acceptance criteria
acquisition phase 101
activation phase 101
activity ratio 145
advanced testing, in environment
acceptance tests (behavior-driven development) 56
container, scanning 56
Dynamic application security testing (DAST) 55
load/performance testing 55
verification 54
Agile 4
emergence 5
Agile Manifesto 6
agile project management tools
planning with 49
Agile Release Train (ART) 15, 16, 31, 67, 154, 283
AI for IT Operations (AIOps) 281
Amazon Web Services (AWS) 53, 117, 278
Andon Cord 113
implementing, principles 113
APM Conceptual Framework 97
Application Configuration Management (ACM) 280
application performance monitoring 97
Application Programming Interface (API) 202
Architectural Enablers 162
ART, system team 63
assisting, with demos 63
end-to-end testing, setting up 63
infrastructure, building for solution development 63
release, facilitating 64
solution integration, spearheading 63
Atlassian cloud outage 112
lessons 112
automation 281
Azure Resource Manager (ARM) 56
B
batch sizes
cost 75
decreasing 73
holding cost 75
transaction cost 75
behavior-driven development (BDD) 83, 202, 216, 218, 219
used, for writing acceptance criteria 204, 205
best practices, production recovery
chaos engineering 250
cross-team collaboration 249
fixing forward 251
immutable architecture 251
proactive detection 249
rollback 251
session replay 250
BizDevOps 275
blue/green deployment 232, 251
build-measure-learn cycle 190
bureaucratic culture 26
business case metrics 265
Business Plan Review (BPR) 29
business units (BUs) 31
C
Centers for Medicare and Medicaid Services (CMS) 111
challenges, product development 2-4
competition 3
customer needs 2
quality, ensuring 3
security and compliance 2
TTM pressures 2
chaos engineering 117
aspects 117
learning debrief, creating 121
cherry-picking 115
CI stage, Continuous Delivery pipeline
collaboration 281
communications lead 123
competency measurements, in SAFe
DevOps Health Radar 169
configuration management
releasing with 54
configuration management (CM) tools 9, 10, 116
Continuous Delivery Pipeline 18, 40, 68
value stream, running through 18, 19
continuous deployment 53
environment, monitoring 56
environments, configuring with infrastructure as code (IaC) 53, 54
releasing, with configuration management 54
releasing, with feature flags 54
Continuous Deployment automation 239
continuous deployment (CD) tools 10
continuous deployment, environment
alerting 57
log collection 57
performance, monitoring/reporting 56
Continuous Exploration (CE) 18
continuous integration (CI) 18, 50
changes, orchestrating 51
packaging, for deployment 52
quality, verifying 51
versus continuous delivery 50, 51
versus continuous deployment 50, 51
Continuous Integration automation 239
continuous integration (CI) tools 10
continuous integration/continuous deployment (CI/CD) 116
automated unit testing 226
building 221
practices, testing 225
static analysis for application security 226, 227
using, to roll forward 125
version control practices 222
continuous integration, quality
static analysis 52
unit tests (test-driven development 52
continuous learning 173, 174, 272
Continuous Security Monitoring (CSM) 262
factors 205
Create, Read, Update, and Delete (CRUD) 213
crisis 274
Crystal method 6
culture
bureaucratic culture 26
for organizational change 26-28
pathological culture 26
Culture, Automation, Lean Flow, Measurement, and Recovery (CALMR) 16, 17
cumulative flow diagram 92
used, for measuring cycle time 94
used, for measuring throughput 95
used, for measuring WIP 93, 94
customer need pivot 193
customer segment pivot 193
customer value
innovation accounting, measurement 192
MVP, building with 191
releases, decoupling by component architecture 256, 257
cycle time 79
measuring, with cumulative flow diagram 94
D
dark launches, using feature flags
example 242
dashboard metrics 265
data artifacts types
performance 96
reliability 96
security 96
database administrators (DBAs) 62
data selection 281
data types
logs 96
metrics 96
traces 96
Definition of Done (DoD) 71
Definition of Ready (DoR) 71
deployment automation 238, 239
Deployment Management 280
development value streams 41, 42
audiences, finding 142
mapping 139
process and workflow, finding 139, 140
emerging, trends 274
for organization 284
people and processes 12
scaling, with Scaled Agile Framework 15
technologies 280
tools and technologies 9
XOps 275
DevOps Health Radar 169
DevOps journey
CI/CD pipeline, establishing 273
deployments, creating 274
environments, creating 274
learning 274
measuring 274
monitoring 274
right to pivot 274
tests, automating 273
value stream mapping workshop 272
value stream, working in small batch sizes 272
DevOps movement 14
aspects 157
DevOps Research and Assessment (DORA) metrics 157
code and documentation, creating 49, 50
planning, with agile project management tools 49
advocacy team 61
as external service 59
as infrastructure as a service 59
container-driven collaboration 62
Dev and DBA collaboration 62
Dev and Ops collaboration 58
fully shared Ops responsibilities 58
SRE team 61
team, with expiration date 60
DevOps transformation canvas 147, 148
characteristics 147
disaster recovery
elements 260
disaster recovery, measurement goal
Recovery Point Objective (RPO) 261
Recovery Time Objective (RTO) 261
disaster recovery, methods
Backup as a Service (BaaS) 262
backups 261
cold site 261
data center disaster recovery 262
Disaster Recovery as a Service (DRaaS) 262
hot site 261
discourse 179
Domain-Specific Language (DSL) 219
specification 219
DORA KPI metrics 157
change failure rate 159
deployment frequency 158
lead time 158
mean time to repair 159
trends, emerging from State of DevOps report 160
dynamic application security testing (DAST) 55, 274
E
eight-step method, for driving transformation of culture 28
new changes, anchoring in culture 31
powerful coalition, guiding 29
sense of urgency, creating 28, 29
short-term wins, generating 30
vision, communicating 30
vision, creating 29
vision, enacting 30
wins, consolidating to drive change 30, 31
enablers 68
architectural 68
compliance 68
exploration 68
infrastructure 68
enabler stories 212
types 212
end-to-end testing 227
equivalent test environments 227
nonfunctional requirements, testing 231
service virtualization 229-231
test data management 229
Environment Management 280
epic 193
error budget 114
espoused theory
versus theory in use 176
Essential SAFe configuration 283
Exploration Enablers 162
Extreme Programming (XP) 5, 214
F
feature flag management 244
use case 244
feature flags 10, 11, 242, 251
releasing with 54
rolling back 125
visibility, releasing with 54
Fit for Purpose (F4P) metrics 103
general health indicators 104
improvement drivers 104
net promoter score 105
vanity metrics 104
Flow Framework® model 161
Flow Framework® model, Flow items
defect fixes 162
features 161
risk avoidance 162
technical debt reduction 162
types 161
Flow Framework® model, Flow Metrics 163
flow efficiency® 165
flow load® 164
flow time® 164
flow velocity® 163
Flow Metrics 161
Forming-Storming-Norming-Performing-Adjourning (FSNPA) model 85
full stack telemetry 96
application performance monitoring 97
infrastructure monitoring 97
network monitoring 98
observability 99
types 54
future development value stream
continuous improvement, checking 147
flow occurrence, checking 146
process, fixing 146
G
Gemba walk 198
General Data Protection Regulation (GDPR) 2
Git 9
GitOps 282
Global Product Development System (GPDS) 30
good measurements
Google HEART framework 102, 103
dimensions 102
Government Accountability Office (GAO) report 111
growth assumptions 265
H
handoffs
setting up 123
healthcare.gov 110
fixing 111
Health Insurance Portability and Accountability Act (HIPAA) 3
Highest-Paid Person’s Opinion (HiPPO) 35
holding cost
versus transaction cost 75, 76
I
IaC files
scanning 55
iceberg model
event level 181
levels 181
mental model 181
pattern level 181
reference link 181
structure level 181
Improvement Kata 181
incident commander 122
incident command post 123
incident management roles 122
communications lead 123
incident commander 122
incident planning/logistics 123
operations lead 123
incident planning/logistics 123
incident state document 123
inference 281
Infrastructure-as-Code (IaC) 240, 274
used, for configuring environments 53, 54
Infrastructure Enablers 162
infrastructure monitoring 97
Infrastructure Orchestration 280
business case metrics 265
dashboard metrics 265
measurement 192
net present value 266
innovation accounting framework 100
Inspect and Adapt (I&A) 268
Internal Developer Platform (IDP) 280
Internet Relay Chat (IRC) 123
IT service management (ITSM) 57
K
Kaizen 7
Kanban 8
Kanban board 68
features 69
impediments, flagging 70
policies, for specifying exit criteria 71
urgent issues, flagging 70
workflow, specifying with additional columns 69
key performance indicators (KPIs) 147, 154
characteristics 155
Kingman’s Formula 79
Kingman’s Formula, variability
process buffers, setting up 82, 83
standard processes, establishing 83
known assets 262
L
launch coordination engineering (LCE) 116, 117
functions 116
lead time 143
Lean-Agile mindset 31
Lean flow
queueing theory, monitoring to enable 77
Lean Improvement Cycle
Lean manufacturing 7
principles 7
Lean User Experience (Lean UX) 198, 199
benefit hypothesis, constructing 199
evaluation 200
Minimum Marketable Feature, building 199
working on design, collaboratively 199
learning organization 173
characteristics 174
mental models 175
Left-Hand Column Analysis 176
Little’s Law 79
load/performance testing 55
M
Marketplace Lite (MPL) 111
mental models 175
espoused theory, versus theory in use 176
reflective practice 176
Minimum Marketable Feature (MMF) 199
Minimum Viable Product (MVP) 100, 167, 190, 267
building with 191
examples 191
mob programming 215
monitoring-as-a-service (MaaS) 57
MPEG audio Layer-3 (MP3) player 3
MVP feature
acceptance criteria 204
beneficiaries 204
benefit hypothesis 204
N
National Institute of Standards and Technology (NIST) 262
negative vision 178
net present value 266
Net Promoter Score (NPS) 155
Network Access Control (NAC) 98
network monitoring 98
Non-Functional Requirements (NFRs) 18, 50, 162, 193
testing 245
Non-Functional Requirements (NFRs) compliance, practices
designing for operations 220, 221
threat modeling 221
O
Obamacare 110
Objectives and Key Results (OKRs) 155
example 155
observability 99
one-button deployment 241
operational value streams 41, 42
operations lead 123
P
pair programming 214
patterns 214
pathological culture 26
pattern discovery 281
people and processes, DevOps 12
learning from failure 13
no fingerpointing 14
trust 13
Percent Complete and Accurate (%C&A) 272
persevere 267
artifacts 174
PI planning
acquisition phase 101
activation phase 101
referral phase 102
retention phase 101
revenue phase 102
pitfalls
avoiding 271
Plan-Do-Check-Adjust (PDCA) cycle 181
phases of activities 183
reference link 280
primary market research 197, 198
process blocks 140
process step metrics 142
percent complete and accurate (%C&A) 143
process time and lead time 143
process time 143
product development 1
product development queues
versus product manufacturing queues 78
production environment
architecting for operations 263
Continuous Security Monitoring (CSM) 262
deploying 238
deployment frequency, increasing 238
failover and disaster recovery 260, 261
full-stack telemetry 246
monitoring 246
NFRs, testing 245
proactive response 249
risk, reducing 241
site reliability engineering (SRE) 259, 260
solution, operating 259
solution, stabilizing 259
test automation 244
test data management 245
versus test environment 230
production failures
production testing
advantages 243
Product Management (PM) 15, 33
Profit & Loss (P&L) 99
Program Backlog
features, prioritizing with WSJF 205-207
Program Increment (PI) 16, 33, 162
Program Predictability Measure (PPM) 167
project-based management
versus product-based management 83-86
project budget 84
project management
factors 84
project management collaboration 196
Agile teams 197
business owners 196
customers 196
product owners 197
system architects 196
project management, research activities 197
Gemba walks 198
Lean UX 198
primary market research 197, 198
secondary market research 198
Q
queueing theory
monitoring 77
queueing theory, to enable Lean flow
R
Recovery Point Objective (RPO) 261
Recovery Time Objective (RTO) 261
referral phase 102
reflective practice 176
regression testing 55
release engineering 115
Release on Demand 19
Release Train Engineer (RTE) 15, 38
retention phase 101
retrospective 182
Return on Investment (ROI) 35, 99
revenue phase 102
rogue assets 263
Role-Based Access Control (RBAC) 280
rollback 251
S
SAFe® Lean Startup Cycle 193-195
SAFe House of Lean 32
foundation 33
roof 32
sanity testing 55
Scaled Agile Framework (SAFe®) 14, 283
adopting, to move to DevOps 283
competency, measuring 168
core values 34
DevOps, scaling with 15
implementation roadmap 132, 133
measurements 166
outcomes, measuring 166
Scrum 6
Search Engine Optimization (SEO) 101
secondary market research 198
Secure Shell (SSH) 54
Secure Software Development Framework (SSDF) 160
self-service deployment 241
service-level agreements (SLAs) 113
service-level indicators (SLIs) 113, 114, 259
service-level objectives (SLOs) 113, 114, 259
parts 113
service virtualization 229
factors 229
session replay 250
benefits 250
set-based design (SBD) 36
shared metrics 11
shifting left 215
Site Reliability Engineering (SRE) 113, 259
site reliability engineers (SREs) 61
smoke testing 55
Software as a Service (SaaS) 57, 100
software development life cycle (SDLC) 276
solution, architecting 200
operations, maintaining 202, 203
releasability 201
security, designing 201
security designing, practices 201
testability, ensuring 201, 202
solution delivery
blockers and bottlenecks 91
cycle time 90
lead time 90
measuring 89
measuring, with cumulative flow diagram 92
throughput 91
Work in Progress/Process (WIP) 90
solution development, CI stages 212-220
staging environment 231
static analysis 226
static code analysis 226
static security analysis 226
static application security testing (SAST) 226, 273
Statistical Process Controls (SPC) 276
Strangler pattern 202
Supply Chain Levels for Software Artifacts (SLSA) 160
swarming 215
synthetic monitoring 245
synthetic transactions 245
System Architect (SA) 15
system demo 232
T
technology-based DevOps trends
AIOps 281
GitOps 282
Test-Driven Development (TDD) 202, 216-218
test environment
versus production environment 230
testing pyramid 216
tests
quadrants 228
text patterns 226
threats and vulnerabilities
cross-site scripting (XSS) vulnerabilities 263
data leaks 263
domain hijacking 263
leaked credentials 263
man-in-the-middle attacks 263
open TCP/UDP ports 263
poor email security 263
typosquatted domain 263
throughput
measuring, with cumulative flow diagram 95
Time to Restore Service 248
tools and technologies, DevOps 9
automated infrastructure 9
IM robots, on shared channels 12
one button builds/deployment 10
shared metrics 11
total lead time 144
total process time 144
Toyota Production System (TPS) 7, 31
transaction cost
U
unit tests 226
unknown assets 263
User Interface (UI) 199
user interface/user experience (UI/UX) 50
utilization 80
V
value
benefit hypothesis, disproving 266
benefit hypothesis, proving 266
innovation accounting 264
measuring 264
value assumptions 265
value proposition
Fit for Purpose (F4P) metrics 103
Google HEART framework 102, 103
innovation accounting framework 100
measuring 99
pirate metrics 100
Value Stream management (VSM) 131
value stream metrics 144
activity ratio 145
rolled percent complete and accurate 145
total process and lead time 144
context for development, setting 133
development value streams 41, 42
funding 84
implementation roadmap 133
improvement areas, finding 142
mapping session 268
operational value streams 41, 42
organizations mindset, aligning to 131, 132
outcomes, learning 266
pivot or persevere 267
relentless improvement 267
running, through Continuous Delivery pipeline 18, 19
value streams identification
customers perspective, with Gemba walks 135, 136
preparing for 134
scope, determining 134
team, creating 135
vanity metrics 100
accessible 100
actionable 100
auditable 100
vendor assets 263
Verification and Validation (V&V) 68
version control 220
practices 220
version control practices 222
gated commits 225
trunk-based development 224, 225
version control systems (VCS) 9
virtual assets 230
virtual machine (VM) 68
Virtual Private Networks (VPNs) 98
W
Weighted Shortest Job First (WSJF) 35, 203
used, for prioritizing Program Backlog features 205-207
work
synthesizing 203
Work in Progress/Process (WIP) 37, 71, 90
measuring, with cumulative flow diagram 93, 94
work synthesis
acceptance criteria, writing with BDD 204, 205
MVP feature, completing 203
PI planning, preparing 207, 208
Program Backlog features, prioritizing with WSJF 205-207
X
XOps 275
Z
zoom-in pivot 193
zoom-out pivot 193