Chapter 15. The Learning Review

Whenever Bill was running late, he always asked himself “What am I avoiding?” Today, as he slowly walked down the hall toward the conference room, the answer was unmistakable: although he intuitively felt sure about the new way of dealing with failures and outages—grounded in complexity science and human factors—he also wished he could gently ease into this new world with a little less at stake. Instead, he was headed toward what he knew would be a very difficult conversation, revisiting the biggest outage in recent history. His companions on this journey were also recent converts, inexperienced and unaware of the potential shortcomings of this approach. “Don’t try this at home, folks,” he thought as he opened the conference room door, and saw Mike seated on one side of the table, and Ollie, Raj, and Linda on the other.

“Glad you could join us,” Raj said, flashing a mischievous smile.

Bill sat down next to Ollie. “Sorry I’m late.” He cleared his throat. “First of all, Mike, thanks for being here.”

Mike nodded.

“We wanted to redo the postmortem of the network outage, and learn more about what happened. We’re calling this the ‘learning review,’ and it’s going to be a bit different from the postmortems we’ve done before.”

“Well, for one, you can’t fire me,” said Mike.

“I’m glad you brought up the elephant in the room,” said Linda. “Are you worried that this learning review will turn into another blame session?”

“There’s certainly a track record of just that,” Mike said.

“It’s hard to be open with so much at stake,” Linda said.

“Yes. But, look, even without all that, it’s not exactly my idea of a fun time to dig through the rubble of outages past, especially ones that I’ve caused.”

“So you don’t want to be here?”

“I’m here mostly because Bill asked me to be here. I honestly don’t know what I can do or say that I haven’t already said or done. I don’t know what you’re after, exactly.”

“So you feel forced into a situation that seems hopeless and pointless?”

Mike paused for a moment. “Yes, that pretty much sums it up.”

Linda took a breath. She looked directly at Mike and nodded slightly.

“I just don’t like being on trial,” he said after a long silence. He exhaled loudly, letting his shoulders relax a bit.

“Believe me,” Ollie said, “I don’t like being the jury. And look, we’re even seated like it’s some kind of congressional interrogation.” He stood up, and rolled his chair to Mike’s side of the table. Linda, Raj, and Bill followed, completing a circle.

Mike looked right and left, with a slight smile. “We gonna sing ‘Kumbaya’ now?” Everyone laughed. The room felt more relaxed.

Finally, Ollie said, “Mike, thank you again for being here. I know it’s not easy. We’re trying something new, and I hope you’ll humor us.”

“How is this learning review different from a postmortem?” Mike asked.

“Fundamentally, in a learning review, we recognize that we’re most likely working with a complex system, which requires a different approach than working with other types of systems. We know that in complex systems, the relationship between causes and effects can be teased out only in retrospect, if ever. We’re also not looking for a single root cause. Instead, we hope to understand the multitude of conditions, some of which might be outside our control. We also accept that some of the conditions will remain unknown and unknowable.

“Also, so many times in the past, our search for the root cause has led us to the individuals we thought caused the incident. But in the learning-review framework, if we ever get to a single root cause—especially if it’s a person—we know we’re not digging deep enough.”

“And, so we don’t take a shortcut to blame,” Raj added. “Let’s remember that the single root cause of all functioning and malfunctioning systems is impermanence. If you’ve got to blame something, blame impermanence.”

“That’s right,” said Ollie. “And to make sure that we don’t take that shortcut to blame, we treat anything disclosed during a learning review as protected information—meaning it won’t be used against the people who share it.”

“Where were you when I needed you? That’s pretty revolutionary, if it’s true,” said Mike.

“It is revolutionary for us,” said Bill, smiling. “And we’ve got Roger’s support for it. With learning reviews, we’re hoping to make our systems more resilient. This requires full accountability, which requires going beyond blame and punishment.”

“We also want to learn not just what went wrong,” Ollie said, “but what went right—what usually goes right. We’re typically overly focused on failures, forgetting that the same systems—including the people working in them—produce both positive and negative outcomes. Mostly positive, in fact—we certainly don’t have outages every hour or even every day!”

“And finally,” Ollie continued, “we’ll be mindful of cognitive biases, which will invariably show up. They’re dangerous because they’ll make us jump to conclusions too quickly, oversimplify things, and to make incorrect causal attributions. Perhaps the most prominent one is hindsight bias. Biases are pretty hard to spot in oneself, but easier to see in others.”

Ollie looked around the circle. “Are we all OK with naming the biases when we see them?” Everyone nodded in agreement. “OK, then, with your permission, I’ll be the facilitator. Bill’s going to take notes.”

Bill got up and walked over to a whiteboard.

“You did always jump at a chance to breathe in marker fumes,” Mike said, laughing.

“Well, you know, every little bit helps in our line of work,” said Bill, closing his eyes, and taking a long whiff of the uncapped black marker. “All right,” he said, still laughing, “let’s get started.” He quickly drew a timeline on the whiteboard.