The Scrum Framework is built on an extensive theoretical base that ranges from systems analytics to team interactions and human behavior. Ken Schwaber and Jeff Sutherland put the pieces together during the 1990s, but its theoretical roots go back to at least the 1980s and, arguably, back to the 1950s as well.
Scrum’s organizational aspects are based on the work of Hirotaka Takeuchi and Ikujiro Nonaka, two Japanese academics who studied teamwork at the Toyota Motor Company. They published their findings in 1986.1 These two scholars were the first to use the word Scrum to describe team behavior. Their observations at Toyota reminded them of a play in rugby football. During the rugby “scrum” each team huddles together in a big cluster to push the ball forward. This idea of “the whole team pushing together” is what Takeuchi and Nonaka found to be the most compelling aspect of the way effective teams worked together at Toyota.
Scrum’s artifacts and events are based on the theory of Empirical Process Control. This theory draws on the work of W. Edwards Deming and others; it holds that decisions should be made based on measurements of actual facts as opposed to predictions of future results. Empirical Process Control holds that it is impossible to predict future events and outcomes with any degree of certainty. It is not possible to be certain about future realities until after they have taken place, not before. The past is real. The future is a mystery and predictions of it are only guesswork.
Transparency : In this context, transparency means that every aspect of a product and the work of developing it should be visible and accessible to everyone who is involved. In other words, everything about the product should be on the table in plain sight. There should be no secrets, and all that is known about the product and its development should be examinable by everyone who has a stake in the outcome.
Inspection : If there is transparency, then the people involved with the product have a duty to examine that transparency on a regular basis. The product and the work being done on it need to be inspected formally on a regular schedule.
Adaptation : If there is transparency and a product is being inspected, then there ought to be lessons that are learned as a result. The principle of adaptation turns the formal inspection of transparency into a feedback loop in which lessons are learned from examining observed facts.
The “Empirical” part of Empirical Process Control comes from the emphasis on using observations instead of predictions. It turns out that the three core principles encapsulate the scientific method used in experimental sciences. In physics or chemistry, when something is to be learned, an experiment is constructed to test a hypothesis. The experiment is designed to produce clear results (transparency), which are then observed (inspection) and used to confirm, disprove, or alter the hypothesis (adaptation).
The Scrum Framework uses this scientific method approach to deal with the unknowns that are inherent in complex adaptive problems. Large problems are broken down into smaller pieces, and a few high-priority pieces are then implemented. After that work is done, the entire problem is reevaluated based on lessons learned. A new set of high-priority pieces are then selected, the work of implementing then takes place, and the process repeats.
This “experimental” approach to software development gives rise to the most visible aspect of the Scrum Framework. With Scrum, work is divided into two- to four-week Sprints. Each sprint has its own “scope,” or set of functionality to produce.
The reason for dividing work into discrete sprints is to have some real results to measure. After each sprint, there are some new pieces of empirical data to add to our knowledge of the problem we are solving. When one sprint ends, the next one begins. The first activity in the new sprint is to create a plan for what to do next. The new plan takes into account any lessons learned from the previous sprint.
When setting out to create a new product using the Scrum Framework, an overall goal is set. A sprint is then planned and executed to take the first step toward that goal. After that sprint is completed, the lessons learned are evaluated and a second step toward the goal is taken. We keep going in this step-by-step process until we decide it makes no sense to continue. As long as there is value in doing so, we keep on going.
How Effective Is the Scrum Framework?
The main reason that organizations are interested in adopting the Scrum Framework is that there is plenty of evidence that it works very well. There is a famous example of the relative benefits of using a Scrum approach instead of a traditional approach. The example is the Sentinel project undertaken by the FBI in the mid 2000s. To understand the project, it is necessary to understand its background.
In summer 2001, the Phoenix, AZ, office of the FBI took note of some suspicious activity. A number of young men from the Middle East were taking flying lessons. Phoenix has a school for learning to fly large commercial jets, and these people had enrolled. They were especially interested in learning how to fly jets in level flight. They didn’t seem to care much about taking off or landing. The FBI watched these individuals for a while, but they then vanished. They were no longer in the Phoenix area.
They later turned up in Boston, where they boarded a number of airplanes at Boston’s Logan International airport. The date was September 11, 2001. These people flew two of the jets into the Twin Towers of the World Trade Center in New York. Another jet crashed into the Pentagon in Washington, DC. The fourth jet crashed in Pennsylvania after the passengers realized what was happening, attacked the hijackers, and gave their lives downing the airplane. It was the worst terrorist attack in US history.
In the aftermath of the attack, a congressional inquiry was held. One of the observations was that while the Phoenix office of the FBI was watching these individuals, the Boston office of the FBI knew nothing about them. It was decided that never again should the United States be attacked, in part, because one office of the FBI had information that another one needed and did not have. There was to be a single database and repository to consolidate all of the FBI’s information so that every office would have access to all information available, no matter what the source.
Thus, the Sentinel project was born. Congress appropriated $450 million to get the job done and set a deadline of two years. Lockheed-Martin, a famous defense contractor, was engaged to manage the project. Lockheed hired 200 developers and got to work,
Eighteen months later, Lockheed-Martin came back to the FBI to report progress. Unfortunately, there was not much tangible progress to report. Some functions were complete but most of the work was yet to be done. Lockheed-Martin asked for more time and more money.
As a tax payer, I’m very grateful to the FBI for what they did next. They told Lockheed-Martin, “You’re fired.” They then brought in some “agilists” who reorganized the effort using the Scrum Framework.
The first thing the agilists did was drop 180 of the 200 developers. They proceeded with only 20 of the original people that Lockheed-Martin had hired.
What was the result? Those agilists and the 20 developers delivered the entire working product only five months later. Twenty people had done in five months what 200 people had been unable to do in 18 months.
Some simple arithmetic shows how dramatic a difference the Scrum Framework made. Just using the number of developers as a gauge, those 20 people were ten times as effective as the original 200 people who were hired. That is a 1,000% improvement.
When the difference in elapsed time (5 months vs. 18 months) and other factors are taken into account, the effect of using the Scrum Framework works out to be a 1,500% improvement.
Is it any wonder that so many organizations are interested in adopting the Scrum Framework?
How could 20 people do what 200 people could not? The 20 people were working in an organized way, whereas the 200 people were working in a disorganized way. The difference was organizing the work using the Scrum Framework as opposed to the traditional “waterfall” approach. Choosing the correct framework for organizing a project can make a huge difference.
Summary
The Scrum Framework relies on two main bodies of theory. It uses the ideas of Hirotaka Takeuchi and Ikujiro Nonaka as the basis for its roles and teams. It uses the theory of Empirical Process Control to give those teams the tools they need to manage the work. The result is a way of organizing people that is simple and very effective.