INDEX
Activities, serial vs. parallel, 71
Adages (Erasmus), 66
Adverse selection, 232
Aerobus, 41
Aerospace sector, xi, 32–33, 233
Aggregation of risk, 90–99, 156–158, 162, 219–220
Agile approach (software development), 139
Airbus A380 airplane, 53, 136–137
Arrow, Kenneth, 58
Asimov, Isaac, 146
Auburn University, 121
Australia, 148
Avatar (film), 86
B-2A stealth bomber, 211
Banking industry, 247
Baron-Myerson mechanism, 237–238, 237f, 238t
Baseball, 62–63
Batting averages, 62–63
The Battle for Investment Survival (Loeb), 144
Belief, in God, 177–178
Bell, Alexander Graham, 143
The Bell Curve (Herrnstein and Murray), 62
Bidding, contract, 24–25
Big data, 65–66
Billionaires, 87
Binet, Alfred, 62
Bivariate distributions, 115–116
The Black Swan (Taleb), 85, 123
Black swans, xv, 55, 105, 227, 242, 243
Black-Scholes model, 140
Boeing 737 MAX aircraft, xi, 41, 42
Boeing 747 aircraft, 53
Boeing Corporation, 24
Bohr, Niels, 123
Bol, Manute, 81
Book sales, 85
Boston Red Sox, 63
Box, George, 109
Brett, George, 63
Bubonic plague, 57
Budget constraints, cost growth due to, 44–46
Budget wedges, 204
Buffett, Jimmy, 105
Bureaucratic issues, 33–34, 46
Bush, George H. W., 38
Bush, George W., 106
Cabrera, Miguel, 63
Calibration, 124–132, 125f, 246
California Department of Motor Vehicles, 34
California High-Speed Rail (CHSR), 17–18
Cameron, James, 86
Cancer, 64
Candide (Voltaire), 1
Canute, King, 32
Capacity, excess, 51
Carew, Rod, 63
Cassini mission, 16
Central limit theorem, 133–134
Central tendency, measures of, 64, 85
Certainty, 174
Charlie Brown, 23
Chebyshev inequality, 172–173
Chernobyl nuclear disaster, xi
Chess-playing computers, 8
Chief risk officer, 229
Child, cost of raising a, 45
Church of England, 59
Churchill, Winston, 140
Citibank/Citicorp, xiv
Cities, population of, 85
Clarke, Arthur C., 146
Coefficient of variation, 170
Coefficients of variation, 127–129, 128t
Coherent risk measures, 181–183
Cold War, 215–216
Colorado Rockies, 63
Comanche helicopter program, 9
Commercial off-the-shelf (COTS) software, 70
Concorde supersonic airplane, 12, 136, 138
Cone of uncertainty, 21–123, 122f, 123f
Confidence intervals, 124
Confidence levels, 95–96, 101–102, 130t, 245
definition of, 95
and funding, 148–150, 156–160, 160t
in joint cost and schedule risk analysis, 224–226, 224f, 225f
in quantitative cost risk management, 216–221, 218f, 219t
tails and limitations of, 173–177, 175f, 176t
Consequence (of risk), 14–15, 74–77, 75t
Context of risk management, 226–227
Contingency, 223
Continuous distributions, 78
Contractors:
cost data held by, 233–236
and fixed-price contracts, 231–232
optimism of, 24
rough-order-of-magnitude estimates of, 24
Contracts, cost-plus vs. fixed-price, 25, 230–232
Cootner, Paul, 140
Correlation, 66, 111–117, 113f, 114f, 117f, 118f
Cost and schedule risk analysis:
and aggregation of risk, 90–99, 219–220
correlation in, 111–117, 113f, 114f, 117f, 118f
definition of, 60
and estimation of uncertainty, 106–108
and evolution of S-curves, 118–120, 119f
issues in, 245–250
lognormal distributions used for, 89–90
probability distributions for (see Probability distributions)
sensitivity analysis, 99–102, 100f, 101f
theory vs. practice in, 120–124, 122f, 123f
track record for, 108–110
Cost growth (cost overruns), xi–xii, 2–3, 5–7, 9–19
from black swans, 55
from budget constraints, 44–46
comparison of, in various industries, 13–14, 14t
in development phase, 5
and diminishing returns, 47–48, 47f
from errors in estimation, 35–37
and external dependencies, 50–55
factors in, 21–22
with government projects, 3
insurance protection against, 155
with megaprojects, 11–12
with NASA/DoD programs, 6–7, 6f, 9–11, 10f, 11f
with Olympic games, 13
from poor execution, 32–35
from poor planning, 29–32
scope reduction as response to, 15–19
with software/IT projects, 12–13
with Sydney Opera House project, 29–30
and unrealistic optimism, 22–27, 242
from use of immature/undefined technologies, 27–29
(See also Cost and schedule risk analysis)
Cost growth caps, 154–155, 154t
Cost reports, 233–234
Cost risk, 60–61, 61f, 82, 126 (See also Cost and schedule risk analysis)
Cost risk reserve phasing, 222, 222f
Cost uncertainty, 69–70
Cost-plus contracts, 25, 230–232
Costs:
direct vs. indirect, 33
variable, 209
Cost-sharing contracts, 232
COVID-19 pandemic, xv, 13, 35, 55, 241
Cramer, Jim, 145
Critical Design Review, 119
Critical path, 223
De Moivre, Abraham, 80
Deep Blue, 8
Deepwater Horizon oil spill, xi, 105, 106
Demand, predicting, 50–53
Denver International Airport, 44
Department of Defense, xv, 7, 9, 31, 32, 35–37, 46–48, 56, 211, 226
Department of Energy, 148
Dependencies, external, 50–55
Derivatives, 153–154
Development phase, 4
Diminishing returns, 47–48, 47f
Direct costs, 33
Discover magazine, 64
Disposal phase, 5
Distributions, probability (see Probability distributions)
Diversification, 143–163
The Doctrine of Changes (de Moivre), 80
“Drinking the Kool-Aid,” 23
Ducats, transport of, 67–68, 69f
Economics, 120–121
Education costs, 121
80/20 rule, 86
Einhorn, David, 174
Elephants, 82
England, 58–59
English Channel tunnel, 12
ENIAC computer, 93
Enterprise resource planning (ERP), 18–19
Enterprise risk management, 228–229, 241–242
Ephesus, 229
Equivalent software lines of code (ESLOC), 12–13
Erasmus, Desiderius, 66
Estimates, rough-order-of-magnitude, 27
Estimation, errors in, 35–37
Euclidean geometry, 84
European Space Agency (ESA), 10–11, 16–17
“Everything Goes According to Plan” principle, 107
Exceptional variation, 173
Excess capacity, 51
Execution, cost/schedule growth due to poor, 32–35
Expected shortfall, 186-187, 186t, 219–221, 220t, 226, 248
and portfolio effect, 190–191, 191t
Expected value, 64, 66–68, 70, 73, 76, 90, 178, 184
External dependencies, 50–55, 243
External factors, in cost/schedule growth, 22
Extreme events, 77 (See also Black swans)
Extremistan, 85, 86, 132, 137, 139, 156, 246
Exxon Valdez oil spill, xi
“Faster-better-cheaper” policy (NASA), 38–40, 200
Fat-tailed distributions, 86
Fermat, Pierre de, 58
A Few Good Men (film), 142
Fichte, Johann, 145
Fields Medal, 59
Film profits, 86
Financial crisis of 2008, 73, 241
Financial markets, 139–140 (See also Stock market fluctuations)
Fixed-price contracts, 25, 230–232
The Flaw of Averages (Savage), 63–64
Flyvbjerg, Bent, 11–12, 60, 149, 150
Ford, Gerald, 106
Fortuna (Roman goddess), 57
Forward pricing rate agreements, 37
France, 58
“Free lunch,” 145–147
Friedman, Milton, 120, 121, 146–147
Fukushima Daiichi nuclear disaster, xi, 55
Full House (Gould), 64
Funding constraints, and portfolio planning, 202–211, 203f, 205t, 207f, 207t, 208t, 210f, 211f
Funding reserves, 216–223, 227, 247
Gains, reactions to prospective, 26
Galileo mission, 16
Game theory, 232–233
Gates, Bill, 85
Gauss, Carl, 80
Gaussian distributions, 80–85, 81f, 83t, 97–99, 98f, 99f, 132–134, 148, 150, 150t, 244–245
and expected shortfall, 186–187, 186t
lognormal distributions vs., 179–181
semi-standard deviation vs., 188–189
tails in, 167–168, 167f, 169t, 171–173, 171f, 174–177, 175f, 176t
Giraffes, 82
Glenn, John, 33
Goddard Space Flight Center, 40–41
Gould, Stephen Jay, 64
Government Accounting Office (GAO), 43
Government economic policies, 146
Government project(s), xiv–xv
and contract bids, 24–25
and cost overruns, 3
estimation problems with, 35–37
example of failed, 30–32
execution problems with, 32–35
incentives in, 229–239
misalignment of cost/schedule/technology in, 37–48
portfolio effect for, 151
risk in, 73–74
Government shutdowns, 46
Gradient allocation, 221
Grading on a curve, 80
Graunt, John, 59
Great Depression, 146
Greedy algorithm, 204, 208–209
Gwynn, Tony, 63
Hamlet (Shakespeare), 140–141
“Haste makes waste,” 37–43
Healthcare costs, 121
Heavy-tailed distributions, 86
Height (of people), 81–82
Heinlein, Robert, 146
Herrnstein, Richard J., 62
Hofstadter, Douglas, 7
Household wealth, 86–87
Hubble Space Telescope, xi, xii, 30–31, 38, 54–55, 135
Hunt-Lenox globe, 165
Hydroelectric projects, 13, 34–35
Hypersonic weapons, 74
IBM, 8
Immature technologies, cost/schedule growth due to, 27–29, 243
Incentives, 223, 229–239, 249–250
Indirect costs, 33
Inflection points, 168 (See also Pain points)
Information technology projects, 12–13, 19, 128, 139, 201 (See also Software projects)
Initial costs, 45
Initial planning phase, 4
Initial year, 196
Integrated circuits, 49
Integrated risk management framework, 226–229, 228f
Intel Corporation, 49
Intelligent quotient (IQ), 62, 81
Internal factors, in cost/schedule growth, 21–22
Internal Revenue Service (IRS), 88
Investment portfolio, 143–145
Investment risk, 72–73
Iron Bowl, 121
Iso-confidence level curves, 225–226, 225f
James Webb Space Telescope (JWST), xii–xiii, 135, 138
The Joint Agency Cost Schedule Risk and Uncertainty Handbook, 90
Joint cost and schedule risk analysis, 223–226, 224f, 225f
Joint Strike Fighter (F-35), 35–37, 43, 52
Jones, Jim, 23
Jonestown massacre, 23
Journal of Petroleum Technology, 124
Juran, Joseph, 86
Kansas City Royals, 63
Kasparov, Garry, 8
Keillor, Garrison, 32
Kelly, Walt, 241
Kennedy, John F., 43
Keynes, John Maynard, 144, 146
Keynesian economic policies, 146, 147
K-standard deviation events, 83–84, 83t
Las Vegas Factor of Development Program Planning, 7, 9
Launch vehicle availability, 53–55
Law of Marginal Survival, 211
Leadership, 246
Learning (manufacturing), 51
Leibniz, Gottfried von, 1
Lemons (used cars), 232
Lewent, Judy, 229
Liberalism, 146
Littoral Combat Ship (LCS), 52
Lockheed Martin, 7
Loeb-Magat mechanism, 235–238, 236f, 238t
Logarithms (logarithmic scale), 87, 88, 87f
Lognormal distributions, 89–90, 90f, 97–99, 97f–99f, 134–140, 135t, 151, 245
and expected shortfall, 186–187, 186t
tails in, 170–171, 174–177, 175f, 176t, 178–181, 179f, 181f
Lognormal paradox, 178–181, 179f, 181f
Long-Term Capital Management, 73
Losses, reactions to prospective, 26
Lotteries, 59
Magnus, Olaus, 166f
MAIMS principle, 3, 38, 216, 240
Mandelbrot, Benoit, 84–85, 139–140
Manhattan Project, 94
Manufacturing costs, 51
“Margaritaville” (song), 105
Marginal cost, 234–238, 235f–237f
Marginal utility, 26
Market design, 233
Markowitz, Harry, 143, 144, 147–148
Mars, manned missions to, 28
Massachusetts Institute of Technology (MIT), 140
Mean (statistical term), 2, 81, 86, 91, 91f, 127
Measures of central tendency, 64, 85
Mechanism design, 232–233
Median (statistical term), 2–3, 81, 91, 91f, 127
“The Median Isn’t the Message” (Gould), 64
Mediocristan, 85, 132–133, 137
Mensa, 172
Merck Corporation, 229
Merrill Lynch, 174
Ming, Kong, 215
Minnesota Twins, 63
Misalignment of cost, schedule, and technology, 37–48
Mitigation, risk, 227
Mode (statistical term), 3, 81, 91, 91f
Monopolies, 234–238
Monotonicity, 182
Monte Carlo simulation, 93–94, 97, 125, 126, 151, 245, 250
Montreal Olympics, 13
Moon, manned missions to the, 28, 43
The Moon Is a Harsh Mistress (Heinlein), 146
Moore, Gordon, 49
Moore’s Law, 49–50
Moral hazard, 230
Mortality, 58–59
Mortgage default risk, 115
Multiyear contracts, 233
Mumford, David, 59
Murray, Charles, 62
Mutually assured destruction, 215–216
Naive approach to risk analysis, 126, 129–132, 129f, 131f
Narrative fallacy, 123
NASA, xi, xii, xiv, xv, 4, 149
average project cost growth at, 6–7, 6f
charter of, 28
cost growth data for, 9–11, 10f, 11f
cost/schedule tracking by, 4
“faster-better-cheaper” policy at, 38–40, 200
Hubble Space Telescope, xi, xii, 30–31, 38, 54–55
human computers used by, 93
joint cost and schedule risk analysis used by, 223, 226
mission failures at, 40–41
and Moon mission of 1960s, 43
performance vs. cost/schedule at, 48
right tail in missions of, 135
scope cuts at, 16
Space Shuttle program, xi, 5, 15–17, 28–29, 54–55
National Association of Insurance Commissioners, 186
National Health Service, 201
National Missile Defense, 136, 200
National Public Radio, 32
Nature, randomness in, 84
Negative portfolio effect, 156–162, 158t, 161t
Net worth, 85
New York Public Library, 165
Newfoundland and Labrador, 34–35
“News from Lake Wobegon,” 32
Newsom, Gavin, 17–18
Niagara Tunnel megaproject, 148
9/11 terrorist attacks, 55, 139, 241
Normal distribution, 80–81, 81f
North American Aviation, 24
Nuclear weapons, 93, 94, 215–216
O Fortuna (Orff), 57
Obsolescence, 48–50
Office of the Undersecretary of Defense for Acquisitions, Technology, and Logistics, 149
Oilfield projects, 13
Operations phase, 5
Optimism, unrealistic, 1–2, 22–27, 55–56, 110, 242, 250
Orange County, Calif., 73, 153, 244
Orff, Carl, 57
Oversight costs, 33–34
Oxford University, 11
Pain points, 168, 169f, 170–171
Parallel activities, 71
Parallel postulate, 84
Pareto, Vilfredo, 86
Pareto distributions, 86–89, 88f, 99, 135–138, 135t, 138t, 161, 174–177, 175f, 176t, 244, 245
Pareto principle, 86
Parkinson, C. Northcote, 9
Pascal’s wager, 177–178
“Pay it forward,” 146
Pensées (Pascal), 177
Peoples Temple cult, 23
Percentile funding, 163, 166–171, 247–248
Percentiles, 95
Performance, and cost/schedule, 47–48, 47f
Pitt, Brad, 121
Planning, cost/schedule growth due to poor, 29–32, 38
Plato, 120
Pogo (comic strip), 241
Point estimates, 60
Portfolio effect, 132–133, 147–162, 219, 247, 248
and adjusting assumptions to match reality, 151–154, 153t, 154t
and capping cost growth, 154–155, 154t
and confidence levels, 148–150
general failure to utilize, 155–156
reverse, 247–248
Portfolio planning, 195–213, 248–249
and focus on initial year, 196
and funding constraints, 202–211, 203f, 205t, 207f, 207t, 208t, 210f, 211f
and reserves, 219–223, 219t, 220t, 227
and schedule changes (zugzwang), 198–201, 199f, 202f
Portfolio theory, 143–145
Positive economics, 120
Positive homogeneity, 182
Positive semi-deviation, 187-189, 188t, 221, 221t, 226
Positivism, 120
Power law, 85, 86, 88, 103, 134–135, 137, 139, 161
Powerball lottery, 59
Practical Schedule Risk Analysis (Hulett), 102
Prairie Home Companion, 32
Private sector, xiv
Probability:
definition of, 60
as field, 58
Probability distributions, 77–90
Probability Methods for Cost Uncertainty Analysis (Garvey), 102
Production, achieving better values in, 232–239
Production phase, 4–5
Professor Pangloss (literary character), 1
Profit maximization, 120–121, 230–231, 234–235, 235f
Project managers, optimism of, 1–2, 24
Project phases, 4–5
Prospect theory, 26
Public projects, 27, 45–46, 51
in ancient Greece, 229
risk in, 73–74
(See also Government project[s])
Pudd’n’head Wilson (Twain), 143–144
Quantitative risk management, 249–250
with costs, 216–222
with schedule, 223
Random sampling, 93–94
Randomness, 84
Rate effect, 51
Reagan, Ronald, 147
Red River flood (1997), 64
Relative risk, 127
Religion, 177–178
Republican policies, 147
Reserves:
schedule, 223
Reverse portfolio effect, 247–248
Risk(s):
aggregating, 90–99, 156–158, 162, 219–220
blindness to, 244–245
concept of, xiv
consequence of, 14–15, 74–77, 75t
definition of, 60
extreme, and negative portfolio effect, 161–162
investment, 72–73
mathematics of, 58–60
mitigation of, 227
in public projects, 73–74
relative, 127
technical vs. resource, xi, xii
terminology related to, 60–62
underestimation of, 110, 245–247
Risk analysis, costs and schedule (see Cost and schedule risk analysis)
Risk aversion, 26
Risk avoidance, xiv
Risk matrices, 74–77, 75t, 245
Risk-taking, xiv
Rough-order-of-magnitude estimates, 27
Rumsfeld, Donald, 106, 125, 150
Russia, 74
Russian roulette, 173
Sampling, 124
San Diego Padres, 63
Santayana, George, 252
Satellites, 4, 53–54, 107, 227
Saturn V rocket, 5
Savage, Sam, 63–64
Scalars, 127
Schedule changes (zugzwang), 198–201, 199f, 202f
Schedule delays, xi, 2–3, 6–19
from black swans, 55
comparison of, in various industries, 13–14, 14t
and diminishing returns, 47–48, 47f
and external dependencies, 50–55
factors causing, 21–22
and hastened schedules, 37–44
and Hofstadter’s Law, 7–9
with NASA/DoD programs, 10–11, 11f
from poor execution, 32–35
from poor planning, 29–32
and scope reduction, 15–19
with software/IT projects, 12–13
with Sydney Opera House project, 29–30
and unrealistic optimism, 22–27, 242
from use of immature/undefined technologies, 27–29
(See also Cost and schedule risk analysis)
Schedule reserves, 223
Schedule risk, 60, 82, 126 (See also Cost and schedule risk analysis)
Schedule uncertainty, 71–72, 71f
Scope of risk management, 227
Scope reduction, 15–19
Scottish Parliament, 136, 138–139
S-curves, 94–96, 95f, 96t, 101, 101f, 118–120, 119f, 125, 225, 250
Second law of thermodynamics, xiii–xiv
Semi-deviation:
and portfolio effect, 191–192, 192t
Sensitivity analysis, 99–102, 100f, 101f
Serial activities, 71
Shakespeare, William, 140–141
Shannon, Claude, 7
Shortfall, expected, 219–221, 220t, 226, 248
Simulations, 93–99
Skew, 3, 9, 14, 81, 82, 89, 148
Software projects, 4, 12–13, 14t, 18–19, 43–44, 69–70, 139
Soviet Union, 215–216
Space Shuttle program, xi, 5, 15–17, 28–29, 54–55
Spurious correlations, 121
Standard deviation principle, 183
Standard deviations, 77, 83–84, 83t, 86–87, 127, 133–134, 152, 187–188, 188t
Standard risk analysis, 129–132, 129f, 131f
Starship Troopers (Heinlein), 146
Statistics (as field), 58–59
STEREO space mission, 24
Stock market crash (1987), 84
Stock market fluctuations, 85, 136
Storytelling, 21
Strategic thinking, 215–240
and achieving better values in production, 232–239
by analyzing cost and schedule risk jointly, 223–226
and confidence levels, 216–221, 218f, 219t, 224–226, 224f, 225f
with cost risk reserve phasing, 222, 222f
incentives and information as factors in, 229–239
integrated risk management framework for, 226–229, 228f
with quantitative cost risk management, 216–222
with quantitative schedule risk management, 223
Subadditivity, 182
Sunk cost fallacy, 18
Swiss Solvency Test, 185–186
Sydney Opera House, 12, 29–30, 136, 137
Systems engineering, 33
Tactical aircraft, 11
Tail dependency, 115–117, 117f, 118f
Tails, 165–193
estimating the size of, 132–140, 135t
and expected shortfall, 184–189, 186t
in Gaussian distributions, 167–168, 167f, 169t, 171–173, 171f
and limitations of confidence levels for risk management, 173–177, 175f, 176t, 247–248
and lognormal paradox, 178–181, 179f, 181f
and pain points, 168, 169f, 170–171
in Pareto distributions, 86, 88
and portfolio effect, 190–192, 191t, 192t
and risk measures, 181–185
and semi-deviation principle, 187–189, 188t
Taiwan, xv
Technical risks, xi, xii, 242–243
Telephone, 143
Texas Instruments, 51–52
The Theory of Investment Value (Williams), 144
There’s No Such Thing as a Free Lunch (Friedman), 147
Thesis-antithesis-synthesis cycle, 145
Tokyo Olympics, 13
Tornadoes, 121
Traffic, 195–196
Translation invariance, 182–183
Triangular distributions, 78–80, 79f, 174–177, 175f, 176t, 244
Troyer, Verne, 81
Twain, Mark, 143–144
Ulam, Stanislaw, 93–94
Uncertainty, 57–58
assessment of, 244–245
and averages, 62–65
cost, 69–70
definition of, 60
estimation of, 106–108
terminology related to, 60–62
Undefined technologies, cost/schedule growth due to, 27–29
Uniform distributions, 78
Units, number of production, 4–5
“Universal Fantasy Factor,” 7
University of Alabama, 121
Unknown unknowns, 106–107, 125
Up-front costs, 45
US Navy, 52
Used cars, 232
Utility (term), 26
Value at Risk, 148
Variable costs, 209
Venice, Italy, 48–49
Voltaire, 1
Von Neumann, John, xiv
Wadlow, Robert, 82
Walker, Larry, 63
Washington State Department of Motor Vehicles, 34
Wealth, household, 86–87
Weapons systems, 4, 11, 52, 74
Wedges, budget, 204
Whales, 82
Williams, John Burr, 144
Williams, Ted, 63
Word frequencies, 85
Work Breakdown Structure (WBS), 91–92, 92f, 97, 99, 111–113, 113f
World War II, 93
Wright brothers, 143
Wriston, Walter, xiv
Yangping, China, 215
Zipf, George, 85
Zipf’s law, 85