Chapter 16
The Myths and Challenges of Scrum
A studio’s development process is a reflection of its culture, and cultures change slowly. Change usually has to overcome resistance, but change happens—especially in the game industry—whether we like it or not.
Change requires commitment from a studio. It should avoid quick-fix solutions or ritual adherence to defined practices. With Agile adoption, both of these extremes are often seen. Some fall in love with the values and principles and dive into adoption with the illusion that the only possible outcome is success. Others sabotage Agile adoption from the start. Saboteurs repeat “urban legends of Agile failure” or label Agile as “the latest management fad.” This is referred to as spreading fear, uncertainty, and doubt and is addressed later in the chapter.
The Solutions in This Chapter
There is a kernel of truth to many of the myths. This chapter identifies the major ones. It looks behind the myths and explores the truths and falsehoods that they rely upon. It also explores many of the barriers to Scrum adoption. Exposing these facts enables a studio to better judge the value of Agile because the best possible path for adopting it is to understand the reasons behind every practice and leave little to faith. This is the benefit of an empirical system: to base what we do on what we know rather than on theory or conjecture.
Silver Bullet Myths
Common to the tales of vampire slayers, silver bullets possess magical properties that give the hero an advantage when facing certain disaster.
Often the adoption of Scrum is motivated by a disaster. A studio ships a game that is a financial failure or has a project canceled because of budget, schedule, or quality problems. During these times, studio management is more open to change. This isn’t necessarily bad, but desperation often leads them to seek a project management silver bullet.
The problem is that Scrum doesn’t work miracles. It has no magical properties. Improvements in development by using it require an understanding of the underlying principles, not blind faith. Scrum is a framework to build a process that supports talent, great teams, and leaders. It does not replace them.
By avoiding the following silver bullet myths, you avoid falling into the trap of thinking a process can solve all your problems.
Scrum Will Solve All of Your Problems for You
Sometimes we have grand assumptions about what a process can accomplish. Perhaps if we create enough rules, then everyone will follow them, and problems will disappear. The truth is that if there are underlying problems, Scrum will only expose those problems. It doesn’t solve them.
Experience
A former California studio adopted Scrum following another project failure. From the start, the Daily Scrums unleashed a flood of complaints about studio operations. Management quickly decided that Scrum was the source of these problems and immediately halted its use. As expected, the level of complaints fell off, and management felt that they had averted a disaster.
The best that can be asked of Scrum is to facilitate problem-solving through transparency. What is done with this transparency depends on a studio’s culture. This studio closed because it ignored the problems that existed for years.
Fear, Uncertainty, and Doubt
Some myths fall under the category of fear, uncertainty, and doubt (FUD). FUD myths are like urban legends; they are exaggerated stories that have a kernel of truth that make them stick.
There are many reasons people use FUD to sabotage Agile adoption. The main reason is that change alone creates FUD. Change threatens the status quo and therefore a person’s security in his or her position. Sometimes the memory of past successes creates the belief that the practices used then will continue to apply equally as well in the future. Problems are seen not with the process, which has proven itself, but with the developers using it now.
This section addresses some of the common FUD myths about Agile. Chapter 17, “Working with Stakeholders,” shares strategies of overcoming the status quo, which can be the greatest obstacle to adopting Agile.
Endless Development
Agile teams never plan. They just iterate without a goal.
Sometimes a team new to Scrum assumes that being Agile means it doesn’t need to plan. It starts by iterating to discover the game that emerges. Eventually the urgency to ship intrudes, and the team has to scramble, often with enforced overtime, to finish something. This isn’t Agile. It’s an iterative and incremental death march. It’s a danger for Agile teams that don’t sufficiently plan.
Having a shared vision for any project is also essential, whether it is Agile or not. If the Product Owner on a Scrum team does not provide a vision, the team cannot be sure that its work creates the greatest stakeholder value.
Experience
I once knew a team that had no Product Owner. It was working on a first-person shooter. The Product Backlog was prioritized by the lead programmer. Because he was focused on physics, each Sprint would show increasingly impressive physical effects. After the first release, buoyant rag-doll bodies were seen floating down a stream in front of the player. When we asked to see how enemies were shot and fell into the stream, we were told that shooting wasn’t implemented yet.
A few months later, the stakeholders canceled the project.
Management Fad
Scrum is just another management fad. It will be replaced by something else next month.
Someone investigating Agile will encounter labels such as Scrum, Extreme Programming, Evo, RUP, Scrum-#, Scrum Type-C, Lean, feature-driven development (FDD), Kanban, and so on. It’s a confusing process landscape. This was one motivation behind the Agile Manifesto. Decades of discovery about how people best work together to create new products are embodied in its simple values and principles. It is an umbrella “brand” name for any number of processes and frameworks that embrace them.
The expanding list of labels represents the various practices that are constantly emerging as Agile becomes more common and accepted. This book has described numerous practices for game development that lie outside the core definition of Scrum. We could label them (perhaps Scrum-G), but the point is for each studio to evolve and adopt its own practices to best fit its needs. There will never be a single template for game development that applies to even a majority of game types or studios!
Scrum is a great starting place. For almost a decade, people such as Mike Cohn and Ken Schwaber have fought to keep it a simple framework for each team to adapt for the needs of their products:
Agile will go away, but it will most likely go away in the same way discussing the merits of object-oriented development went away. Agile will eventually become the accepted way of doing things, and it will just be what we do. In the same way no one says, “Gotta run, I’m late for the object-oriented design meeting” (they just say “design meeting”), we will stop talking about Agile development but only after it is the norm.
—Mike Cohn
The Double Standard
We hear one of two things about Scrum. Either Scrum was successful, or if the project using it failed, it did so because the teams weren’t using Scrum correctly.
Scrum can’t be blamed for failure or even credited with success. Teams succeed or fail because of many factors: technology, capability, vision, communication, collaboration, talent, or even the underlying idea of the game. Scrum creates transparency into how well these elements are working, often in a measurable way, but that’s all. It doesn’t prescribe what to do when Sprints repeatedly show the game isn’t fun or the velocity of the team is low.
Change Is Bad
Our process has worked in the past. If we change things, we’ll fail.
Studios have a certain level of resistance to change in the process they use to create games. This is not necessarily a bad thing. A studio’s process evolves over many years. It’s reinforced by its strengths and successes.
However, processes also embed cultural weaknesses. For example, many studios institute a one- or two-week “lockdown” before a milestone date. The lockdown ensures that no new features are allowed into the build that can destabilize it while the build is debugged and polished. This is a concession to a development culture that allows too many defects into the game during development.
Lockdowns slow development because the cost of fixing bugs increases the longer you wait. When developers are aware that there is a period of time being set aside for dealing with quality problems later on, they’re less likely to address those problems up front. Also, the entire project staff is usually not engaged in bug fixing during a lockdown, and therefore they have to find other things to do. Lockdowns also end up becoming a dumping ground for checking in bug-ridden code or unpolished assets under the assumption that they will be fixed later. This creates a vicious cycle that can inflate the duration of future lockdowns!
Changing core practices, such as introducing unit testing to improve code quality, is more challenging than adding a quick fix, such as a lockdown. The problem is that quick-fix practices become a normal part of a development.
Over months and years, such quick fixes build up. Each of them adds a bit of drag to development. Resistance to change prevents corrections to the process until there is a catastrophic failure (such as a project cancellation over high cost or slow progress).
Normalization of Deviance
Two examples of how ignoring problems can lead to tragedy are the Challenger and Columbia space shuttle disasters that took the lives of 14 astronauts. Both incidents involved problems that were well known and documented with the shuttle system. These problems had occurred frequently but had not resulted in the loss of a shuttle. As a result, they became an acceptable part of operations until there was an accident.
Following the Challenger accident review, the phrase normalization of deviance was coined (Vaughan, 1996) to describe how the attitude toward defects in the shuttle system resulted in its loss. Unfortunately, identifying this problem with NASA’s culture was not enough to prevent it from contributing to the loss of the Columbia years later.
Endless Meetings
Scrum consists of nothing but meetings!
As previously described, the meetings defined by Scrum within a Sprint cycle are as follows:
Daily Scrum
Sprint Review
Sprint Planning
Sprint Retrospective
All of these meetings, except for the Daily Scrum, occur once a Sprint. Many teams find that for a three-week Sprint, these once-a-Sprint meetings can be conducted in a single day. This represents one day of meetings for a 15-working-day Sprint, or 7 percent of the available time. The Daily Scrum meeting is a 15-minute, timeboxed daily meeting. This uses about 3 percent of the working day. Totaling these percentages results in 10 percent of a team’s time, or a little more than four hours of meetings each week average.
Eliminating the Daily Scrum?
One sign of Agile adoption failure is when teams new to Scrum abandon the Daily Scrum. The typical reason is that they don’t see the value in talking to each other or that the meeting is run in an ineffective way (for example, it’s just a task-reporting exercise).
However, I’ve noticed that some of the most effective teams abandon the Daily Scrum. The reason for it is different: They’ve co-located themselves and communicate throughout the day, so they no longer need the ceremony to improve communication.
Scrum meetings are optimized to create the highest bandwidth of necessary communication. A 15-minute Daily Scrum will often identify issues that are easily solved within a day. If not identified, the issue might grow to waste days and require hours of meetings to address weeks later.
The details in this book will help teams run these meetings as effectively as possible without losing their benefits. If any of these meetings do not engage everyone attending nearly 100 percent of the time, then there is room for improvement. The goal isn’t to reduce meeting time, but to optimize communication effectiveness.
Scrum Challenges
The only thing harder than starting something new is stopping something old.
—Russell L. Ackoff
Scrum is not a “one-size-fits-all” solution for developers. A goal in adopting Scrum is to initiate a never-ending cycle of continual improvement customized to the needs of a studio’s culture, people, and games. Before this goal can be realized, a studio or team has to start practicing the fundamentals and navigating early challenges. This section describes a few of these challenges and some of the ways of navigating them.
Scrum as a Tool for Process and Culture Change
Any process can be used to make games, but no process is perfect. Even if we were to identify a perfect process, the constant change in our industry would quickly make it obsolete. This is a major motive for using Scrum. Scrum is not a process; it’s a framework for creating and evolving your own process. It can help any organization create transparency, which enables commonsense change.
Scrum will influence every part of your studio. Scrum pressures leaders to focus on mentoring and coaching. It involves buy-in from departments such as human resources (HR), which can resist the emphasis on team performance over individual performance. It demands people take ownership in areas in which they had no ownership and give up ownership in areas they once ruled over. It challenges marketing to participate with development far earlier in a project. It even puts pressure on facilities to provide open team areas, wall space. and shared spaces for privacy, when needed.
Scrum adoption starts with creating transparency to expose cultural weaknesses and strengths. An example of this was a studio dominated by technology; games were seen as platforms to demonstrate technical achievement. Tools and pipelines to improve productivity for artists and designers were lower on the list of priorities because the programmers were always trying to accomplish nearly impossible challenges. The build was rarely working because of all the bugs left along the way, so the time it took to iterate on an asset change was very long.
How did Scrum influence change? First, it made the high iteration costs visible. Requiring a potentially deployable version of the game every two to three weeks exposed a lot of problems. At first, the teams spent half their Sprints trying to cobble together a working build. Because velocity was measured by the value of the features working in the game, their initial velocity was low.
This simple measurement of velocity is important. Teams need clear performance measurements to evaluate themselves and make changes to improve that measurement. Our example team did this. They started to come up with ways to improve the reliability of the build. Because velocity derives from what is seen in the game, which includes art and design, they formed cross-discipline teams. They co-located to reduce the overhead of communication. Programmers focused on improving tools and pipelines. All of these things improved their velocity.
Before they adopted Scrum, they would have thought that such change was only possible through great technical effort. Scrum allowed cultural change, which resulted in huge performance gains.
Experience
“After research into methodologies, we were drawn to the advantages of Agile software development and decided to adopt Scrum. Within the first few months of Brütal Legend development, the team was practicing Scrum, and the initial payoffs were impressive. Because of Scrum’s emphasis on features over systems, on rapid prototyping and iteration, on cross-disciplinary teams, on people over process, and on the creation of a potentially deployable piece of software, every Sprint/milestone made the game playable at a very early stage in development.”
—Caroline Esmurdoc, Executive Producer and COO, Double Fine Productions, Inc.
Scrum Is About Adding Value, Not Task Tracking
Running a Daily Scrum is easy. Prioritizing a list of features to be implemented is straightforward. Creating a task board and burndown chart takes minutes. So, why do some teams struggle adopting Scrum?
One reason is that many studios that adopt Scrum come from a task-centric culture. Progress is measured against how well individuals complete their work against estimates. Task tracking is an important tool in Scrum, but value creation takes precedence. If value isn’t maximized, then the tasks were wrong.
Have you ever heard someone report “All the tasks for feature X are completed” only to see that feature X is nowhere near being done? In Scrum, the conversation shifts to discuss a feature’s emerging value and the impact on the tasks for that feature. If this shift doesn’t occur, then the value of Scrum practices is greatly diminished. For example, in a task-centric culture, the Daily Scrum is seen as a task-reporting meeting that is easily replaced. In a culture focused on value and team commitment, the Daily Scrum takes on greater importance.
Cultural change is hard. The status quo will often fight tooth and nail against it. Scrum is a framework for this change, but it comes from real leadership, not from a book or a fixed set of practices.
Experience
“One of the first things I did when starting to work with my current team was this: ‘We’re going to try this Scrum thing. We’re going to start with Scrum, but our goal is to end up with a process that fits this team and the work it does. I will never say no to a change proposed by the team, but I insist on one thing. We will try Scrum by the book for at least two Sprints (one to learn the process and one to figure out what you don’t like about it) before we start making adjustments.’ From day one, it was their process. We were just starting from a template called Scrum.”
—Bruce Rennie, Independent Developer
Status Quo Versus Continual Improvement
Building a culture of “continual improvement” is one of the ultimate goals of Scrum. Scrum practitioners use empirical measures to assess benefits of all practices. If, for example, a practice change improves velocity, then it’s a beneficial change.
This is a simple idea to introduce, but it can be challenging to embed. The status quo, or groundless resistance to change, is often a tremendous barrier to continual improvement.
Fear of change is not always baseless. Management fears that introducing profound change gambles a studio’s future. Introducing change requires bravery. For example, changing a process that may have worked for a PlayStation game paints a big target on the person who changed the process for a live mobile game. There is enough risk in changing platforms alone to cause someone to hesitate over changing one more variable.
This is why it often takes utter project failure and crisis to usher in significant change. This isn’t ideal either because it often leads to the “silver bullet” adoption pattern described previously. Ideally, any method for introducing change needs to do it in ways that
Make small, reversible steps
Are measurable to ensure that any perceived improvement is real
Scrum supports change in this manner. At worst, a change in practices will impact a single Sprint of one to three weeks. Metrics, such as task burndown slope, user story point velocity, or any number of metrics that the team has introduced, provide frequent measurement of the value of changes.
This change of culture can’t happen only from the bottom up. It has to be supported by company leadership. Management needs to understand the tools that Scrum provides them to ensure that teams are making progress and that leadership, vision, and commitment exist.
Lack of management commitment to Scrum and Agile principles is a major challenge for studios attempting to adopt Scrum. Under pressure, a manager might find it easier to alter a team’s goal mid-Sprint rather than fight to preserve the team’s commitment to the Sprint Goal. Educating management about the benefits of Scrum—especially for them—is an ongoing effort.
Cargo Cult Scrum
When teams first start using Scrum, they are encouraged to follow the practices “by the book.” This enables them to become accustomed to the practices while the principles sink in. After the team experiences the initial benefits, it begins altering practices to improve upon those benefits, but it’s important that changes to the practices still preserve the principles behind them.
However, some teams stick to the “out-of-the-book” practices and resist changing anything. These teams end up practicing what I call Cargo Cult Scrum. This refers to the infamous cargo cults of the Pacific.
Until World War II, many Pacific Islanders were never exposed to Westerners or their technology. This changed drastically during the war when the Navy arrived to occupy the islands and construct airfields. When the airfields were complete, cargo planes would land and bring in supplies. These planes carried many things including simple trinkets that were shared with the islanders to promote goodwill. The islanders loved these objects (such as steel knives and mirrors) and gathered around the airfield every time a large aircraft landed seeking further gifts.
When the war ended, most of these airfields were abandoned, and the cargo planes stopped coming. The islanders once again found themselves isolated from the Western world, but they did not forget the cargo planes and their treasures. They wanted them to return. Out of desperation, they tried to draw the cargo planes back by replicating the practices they saw at airfields during the war.
They constructed bamboo towers and manned them with natives wearing coconut headphones who spoke into pineapple microphones. They built bamboo mock-ups of small planes, lit fires next to the runway, and waved flags of cloth in the air. But the cargo planes did not return. The practices weren’t enough to bring them back.
This approach is similar to what is done on Cargo Cult Scrum teams. Following the practices isn’t enough to bring the full benefits of highly productive teams. Teams need to understand the principles behind the practices and improve those practices to best fit their product, people, and culture.
Consistency and Monkeys
Consistency is a hard-coded survival trait. Change is resisted. It’s a primal instinct. I never fully appreciated this fact until I read about the following experiment (Stephenson, 1967)1.
1. There is some debate as to this study being apocryphal. In any case it’s a useful metaphor for what goes on with the methodologies we apply.
Five monkeys were placed in a room with a banana tree at the center. Whenever a monkey attempted to climb the tree to pick a banana, a sprinkler system sprayed all the monkeys with water until the hungry monkey retreated from the tree. They repeated this until all the monkeys learned to avoid the tree.
The next stage of the experiment involved replacing one of the monkeys with one who had never been sprayed with water. The new monkey soon approached the banana tree. However, before it could reach the tree, the other monkeys jumped into action and beat the new monkey until it drew back from the banana tree. This was repeated every time until the new monkey learned not to approach the tree.
The researchers continued to replace original monkeys who had been sprayed with water one by one with fresh monkeys. Eventually none of the monkeys in the room had ever been sprayed with water for climbing the banana tree. The monkeys still continued to beat up any monkey who approached the banana tree, but none of them knew why. They just knew that it was off limits.
Sometimes we developers exhibit similar behavior. A company’s culture becomes intertwined with “best practices” that aren’t questioned and never replaced. Personally, I did this for many years pursuing waterfall methodologies. I wrote big documents and schedules that attempted to address every detail of the project. Even after those projects shipped—following months of crunch and despair—I would start the next project the same way.
Scrum Is Not for Everyone
One of the challenges of Scrum adoption is that Scrum is not for everyone. The initial challenge will be that some people refuse to work in an Agile environment and leave. Some of these people will be valuable. They leave because they are not comfortable with the change in their position.
Some developers reject participation in any team activity (such as a Daily Scrum). Some have grown comfortable from a career of working in relative isolation and being called upon to be heroes during crunch times. Others find a niche in a lax management environment where they are given a great deal of freedom to create technology or assets on their own. Joining a Scrum team and making daily commitments to a team of peers limits these individual freedoms.
Most studios find ways to accommodate these individuals, perhaps creating special “R&D” roles for them, but in some cases they eventually leave. The transparency Scrum introduces to a studio makes such positions stand out and not easily justifiable.
The benefits greatly outweigh the losses. Scrum grows leaders and outstanding contributors at a far greater rate than those who are lost.
Overtime
Scrum doesn’t limit teams to working 40-hour weeks. Its practices enable teams to find a sustainable pace of work. This pace is discovered as they commit to Sprint goals and learn how much they can achieve Sprint after Sprint without overextending themselves.
Note
In everyday vocabulary, a Sprint is something that isn’t sustainable. Calling it a jog might have been more accurate but less appealing.
A Sprint can have many uncertainties. Unanticipated problems or unforeseen work slows progress. When this happens, teams sometimes put in a bit of overtime to fulfill their goal.
How much overtime should Scrum teams work? It’s up to them. If they find that they are working overtime too often, they need to address the problems that are causing it. Common examples for this are late commits or handoffs at the end of a Sprint or committing to extra polishing and tuning work on the last Sprint of a release.
When management doesn’t tell teams to work overtime, their attitude about it changes. When a team decides to work overtime, it does it as a team. Occasionally working a few extra hours shoulder to shoulder with your teammates is a team-building experience.
Crunch
Extended periods of enforced overtime are called crunch. Many studios that are new to Scrum continue to practice it until the empirical measure of velocity demonstrates its futility.
Studies have shown the impact of crunch on productivity and quality of life.2 For High Moon Studios, the proof came when management enforced companywide overtime early in our adoption of Scrum. The teams were told to work 10 hours a day for 6 days a week on a troubled project. The subsequent burndown charts told an interesting story. Figure 16.1 shows the hours the average team burned down per week from their Sprint Backlogs.
2. https://igda.org/resources-archive/quality-of-life-in-the-game-industry-challenges-and-best-practices-2004/
Figure 16.1 Burndown during crunch
Week 1 was a normal workweek, before overtime. Weeks 2 to 5 were crunch weeks of 60 hours each. In the first week of crunch (week 2), velocity greatly increased; more work was being done because of the 50 percent overtime. However, as weeks passed, the velocity decreased until week 5, when the velocity was less than it was before crunch started!
How is this possible? The reasons are simple. People were tired. They made more mistakes. They lost their concentration.
This realization represented a huge benefit of Scrum for High Moon, providing simple empirical evidence about what works and what doesn’t. This is why there is no rule about overtime in Scrum. There doesn’t need to be a rule. If your teams are using Scrum to find the best way to work, they’ll quickly discover that after several weeks of overtime, any benefit from it is lost. It becomes common sense to maintain a sustainable pace.
Velocity in Hours
Although the term velocity usually refers to the measure of scope (for example, story points) accomplished per Sprint, the velocity, or change, in hours of work remaining in a Sprint per day or week provides interesting, though less stable, feedback about a team.
Experience: Counting Cars in the Parking Lot
We first started using Scrum at Sammy Studios. Sammy was owned by a Japanese Pachinko manufacturing company called Sammy Corporation. Although we had gotten off to a slow start, Scrum was helping us get back on track and demonstrate good progress. However, good progress wasn’t enough for Sammy Corporation. A source of concern from them was based on the opinion that American game developers don’t dedicate as much overtime as Japanese developers. They felt the lack of crunch meant a lack of commitment to the success of the studio.
Sammy Corporation eventually merged with Sega to accelerate its transition into game development and publishing. During this time, Sega debated about whether to retain Sammy Studios. To help in the debate, it had someone drive past the studio late at night and count the number of cars in the parking lot. The count was low, which to Sega meant that our commitment to success was also low.
Fortunately for us, there was a newspaper distribution warehouse next door. They were quite busy between midnight and six a.m., so we asked the workers to park in our lot, which wasn’t an inconvenience. This satisfied Sega for a while.
Eventually, though, Sega informed us that it was closing the studio. Fortunately, we were able to acquire the studio, which was renamed as High Moon Studios.
Managers often confuse overtime with commitment. They think that forcing people to work overtime demonstrates commitment to the success of a project. In reality, this is like forcing someone to smile to prove they are happy. It just doesn’t work that way.
What Good Looks Like
The most successful Agile adoptions have a few things in common. One of them is having a leader who is knowledgeable and curious about methodologies, leadership, teaming, and various project management approaches. In other words, having a “process nerd” in a leadership position is essential.
I was the process nerd for High Moon Studios. At first, I read books on methodologies in an attempt to find the perfect one, but after a while I came to realize two things:
Rather than being the latest “buzzword,” Agile is the evolving result of decades of slow change away from the discredited scientific methodology introduced more than a century ago (and still being widely applied as “waterfall”).
The focus of methodology has shifted from process and practices to a focus on people. Growing people, teams, and culture is now recognized as the biggest factors of success.
Desperation and self-survival instincts3 led me to be a process nerd, but having encountered them at thriving studios has led me to realize this position is critical.
3. Motivated by a Sega executive described in Chapter 20, “Self-Organization and Leadership”
Summary
The best way to adopt Scrum is to do it with eyes wide open. Establishing useful metrics and establishing practices such as the Daily Scrum provide immediate demonstration of its value.
Scrum is a framework. It doesn’t include practices to optimize code, create better art, or tune a mechanic. Those come from each studio’s development practices and culture. In this light, there is less to fear about the change Scrum introduces.
Keep the myths and challenges from this chapter in mind as you read the next chapter on launching Scrum, which continues to introduce challenges and describes how to meet them and what to expect in a studio as Scrum principles take hold.
Additional Reading
Heath, C., and D. Heath. 2007. Made to Stick: Why Some Ideas Survive and Others Die. New York: Random House.
Pascale, R. T., M. Milleman, and L. Gioja. 2001. Surfing the Edge of Chaos: The Laws of Nature and the New Laws of Business. New York: Three Rivers Press.