Chapter 6
Kanban
Early in our adoption, Scrum exposed many problems for us to solve. Later we encountered other challenges that presented problems using Scrum for all aspects of video game development:
The team supporting engines and tools for multiple games kept receiving urgent requests that interrupted their Sprints.
The teams that were producing assets such as levels and characters couldn’t cleanly fit their work into timeboxed Sprints.
After some experimentation, these teams decided not to Sprint anymore. Instead, they decided to plan work as needed and release it as completed. This worked well.
This problem had been seen elsewhere before. Jeff Sutherland, one of the founders of Scrum, had encountered this at his company and had called his similar variation “Scrum Type C.”1 Later on, we came across writings by David Anderson and Corey Ladas, who had adapted Lean practices from Japan into a method called Kanban.
1. https://xebia.com/blog/type-c-scrum-explained/
The Solutions in This Chapter
This chapter introduces the basics of Kanban for game development. Later chapters employ those basics for content creation (Chapter 10, “Video Game Project Management”) and to support teams (Chapter 22, “Live Game Development”). It covers
Why Sprints aren’t a perfect match for some types of work
How Kanban visualizes and measures work differently
How Kanban tools help manage work to address bottlenecks and improve flow
The difference between Scrum and Kanban roles and practices
What Is Kanban?
Kanban is a method for visualizing, measuring, and managing the flow of work. It employs many practices for improving that workflow by making problems immediately visible and their impact on the workflow measurable.
Why Scrum and Kanban?
One of the most surprising reactions to talking about both Scrum and Kanban with some people is anger. Scrum and Kanban overlap in principle, and the practices unique to each are complementary. Just as carpenters have different tools for different needs, teams can employ practices that best serve them regardless of the label they fall under.
Visualizing the Workflow
Applying the Kanban method starts with visualizing the workflow of an item. Figure 6.1 shows an example workflow of a character asset.
Figure 6.1 The flow of creating a character
A single-discipline executes each step of the work, and the arrows between represent a handoff of that work to another discipline.
After we capture the workflow of an item, we represent that flow on a Kanban board (shown in Figure 6.2). A Kanban board is a bit similar to a Scrum task board, and is also sometimes called a Heijunka board.
Figure 6.2 A Kanban board
A Kanban board visualizes the current state of the workflow and identifies where it is being disrupted.
A Kanban board has columns that represent each state of work for an asset or feature, which are identified on Kanban cards. In Japanese, the word kan means “card,” and ban means “signal,” so Kanban refers to “signal card.” A Kanban card is a signal meant to signal action, such as “work on this next” or “I need an animation.”
A Kanban card (see Figure 6.3) contains information we want to capture for each stage of work on a character.
Figure 6.3 A Kanban card
In this example, the Kanban card captures the date of completion for each stage of work. This is useful for measuring and smoothing out the flow of work. Kanban cards contribute to the visualization of workflow by showing, at a glance, where work might be falling behind or where someone has nothing to work on.
Measuring the Workflow
Kanban incorporates the same inspect-and-adapt approach to continuous improvement as Scrum. As with any empirical approach, improving something is easier to do if you can measure it.
The primary measurement in Kanban is how long it takes for a work item to go through the entire pipeline. This is called the cycle time. Examples of cycle time for game development are the amount of time it takes for
A character to go from concept to in-game
A tool feature starting from the time the feature was requested to becoming available to the tool user
A proposal for a new mechanic to be deployed to a live game
What’s common with these examples is that Kanban is focused on measuring the entire pipeline. This principle is based on the discoveries in Lean manufacturing that optimizing parts of a workflow often works against the whole of the workflow. For example, if we focus on keeping a character animator busy fulltime, it often results in a large inventory of unfinished characters. This backlog of unfinished work is a debt that can
Lead to long lead times between the concept and deployment of a character
Lead to throwing out those animations when the first character to appear in the game reveals animation problems
Slow iteration time, resulting in fewer iterations and lower quality
Note
Some teams use throughput instead. Throughput is simply the number of things completed in a certain amount of time; so instead of saying a character model took a day of cycle time, the throughput might be five characters per week.
Managing the Workflow
One of the easiest aspects of Kanban initially is that you don’t change how you work when you start it. You just capture your workflow on a board, add some measurements, and continue working as you always have. It doesn’t stay that way, though. Over time you start applying some different ways of managing the work to better highlight the issues that are slowing that work down.
This section describes ways Kanban manages workflow differently, and the next section shows how workflow is improved using these techniques.
Pulling Work
Kanban employs a pull system, where people working “downstream” (work that will be done later) pull work from “upstream” (work that is done earlier). In the example from Figure 6.2, when the animator needs more work, he pulls the next character to be animated from the column containing the previously rigged characters.
This is different from “pushing” work to an animator, usually in the form of a predetermined schedule. The reasons why pulling work is better are
Schedules are more realistic: Work is “pulled” when it hits a quality bar. “Pushing” is usually driven by a fixed schedule and often results in unfinished work that requires late rework, which creates schedule delays. When this initial work items make it through a pull system, we can better forecast the schedule for the remainder of the work.
Quality drives pace: As mentioned, with a push system, work is rushed, and quality suffers. The built-in definitions of done in Kanban (covered later in this chapter) allow a balance to be established between speed and quality that can be continually refined.
Pull systems allow more task ownership: This results in the entire team being more engaged with finding improvements.
Defining Done and Buffering
As each stage of a work item is completed, it moves into a done state for that stage. For that to happen, it has to meet a Definition of Done (see Chapter 7, “The Product Backlog”). For example, with the flow shown in Figure 6.1, for a character model to be considered complete, it has to
Pass an art Review
Meet budget constraints (polygon count, texture limits)
Meet asset naming conventions
If it meets these definitions, it moves into a buffer between it and the rigging stage. This buffer signals to artists doing the rigging that they can pull that model in when they are ready.
Limiting Work in Progress
One of the more powerful techniques of managing work in Kanban is to limit the work in progress (WiP) in every stage and buffer in the workflow. These limits, when reached, trigger a rule (called a policy) that says no more work can enter that stage or buffer. This prevents a pile-up of unfinished work. It signals to the team that it has reached a limit and that something needs to be done, usually resulting in improvements to the workflow (discussed later in this chapter).
Lean Thinking and Boats
A useful metaphor for how Lean thinking encourages continual improvement is that of a boat traveling down a river. The river is filled with boulders of various sizes that impede the flow of water. When the water level of the river is high, the crew sailing the boat doesn’t encounter the boulders and isn’t concerned about how they are slowing them down.
However, when the water level in the river drops, the boulders begin to appear. They are then recognized not only as impediments to the flow of the river, but also as real danger! Now the crew reacts to the boulders and steers around them.
In this metaphor, the boat is the project, and the flow of water represents the flow of money or time being spent on it. The boulders stand for the things that slow development down, such as unreliable build systems or asset pipelines that take hours to create a build.
Lean thinking not only causes the water level to be lowered, through WiP limits and reduced batch sizes, but also arms teams with the equivalent of a cannon, through transparency and self-organization, to start blowing the boulders away. This makes it possible for the boat to go faster and straighter.
Ordered Work
As with Scrum, the team’s work comes from an ordered Product Backlog that a Product Owner manages. For a support team, this allows the Product Owner to balance the urgency of requests coming in.
Planning on Demand
Kanban teams don’t have timeboxed iterations, like Scrum Sprints. Instead of planning a Sprint every one to three weeks, Kanban teams pull a set of work from the Product Backlog when they need to.
Ready Columns
If planning is needed for Product Backlog Items (PBIs) brought in from the Product Backlog, a Kanban team will add a “ready” column. A ready column will have a set of tasks associated with each PBI. An empty ready column is a signal to the team to hold a planning session to refill the column.
Ready columns will usually have a WiP limit that determines how many PBIs to pull in from the Product Backlog.
A Cadence of Review and Retrospectives
As with Scrum, having regular Reviews of the work with the stakeholders is useful to evaluate progress and to have team Retrospectives discuss improvements in how they work. Unlike Scrum, these are not tied to a Sprint timebox. Instead, Kanban teams establish a cadence (rhythm) of Reviews and Retrospectives. The cadence of Reviews is negotiated with the stakeholders based on how long they want to go without seeing what the team has accomplished since the last Review.
The cadence of the Retrospective, in my experience, shouldn’t be longer than once every three weeks. When it’s longer than that, then the urgency of fixing things that impacted the team more than three weeks ago starts to fade.
Optional Cadence
Reviews and Retrospectives can also be held on demand instead of a cadence but be careful not to let too much time pass between them.
Improving the Workflow
After the workflow is visualized and is being measured, everything is in place to inspect and adapt and continually improve it. This section describes the major tools of improvement that teams use. Future chapters on live support and content production explore these tools and others more fully.
Reducing Batch Sizes and Waste
One of the biggest wastes in game development is throwing out assets that were created too soon. This often results from the assets being created before the technology or design assumptions, used to drive asset requirements, are proven to be too optimistic.
Lean development, like Agile, promotes more frequent and low-cost iterations, which gives us more opportunities to find improvements in what we’re making and to ensure we’re making it the right way. By sending small batches of assets through the entire workflow and into a working game, we can avoid this waste.
The same applies to technical work. By sending smaller batches of code through the workflow and proving their functionality, we build up a better architecture rather than just coding to a lengthy technical design document and integrating the work in the distant future.
Reducing Handoffs
Studios often organize developers by disciplines that even sit in separate rooms. These are called silos. Silos work with each other by passing documents. Designers write documents that are “handed off” to programmers who develop architectures and tools that are handed off to content creators who put assets in the game, which is then handed off to testers who then hand off bug reports back to the developers.
Each of these handoffs represents not only a handoff of work, but a handoff of responsibility. When a programmer hands off an architecture, he can feel his responsibility for the game has been handed off as well. If an asset doesn’t work within that architecture, the responsibility for that can be confusing. At best this results in a delay in a solution (assigning tasks to fix bugs can result in many handoffs) or at worst a culture of blame as the artist and the programmer blame the work of the other for the problem.
The Lean principle of reducing handoffs addresses this. As described in earlier chapters, documents are an imperfect means of communication. When teams find ways to be more closely involved with the entire workflow, and not just their own part, the result improves. Chapter 10 explores examples of the benefits of reducing handoffs in content pipelines.
Responding to Bottlenecks
WiP limits create hard stops in the workflow when those limits are hit. For example, if testing has hit its limit of new features currently being tested, it can’t pull in any more. That might create a bottleneck for development upstream. As a result, its own WiP limit is hit, and no more development tasks can be pulled in.
As with Scrum, we first ask the team what to do. Using Lean methodology, it can apply the “Theory of Constraints,”2 which says that you solve the bottlenecks one at a time and wait for the next bottleneck to show up and repeat the process.
2. https://www.Leanproduction.com/theory-of-constraints.html
For the testing bottleneck example, a number of things might be done:
Hire more testers: This solves the problem of testing, but then your testers might run out of things to test, and the programmers then become the bottleneck.
Have the developers stop developing for a few hours and help test: Some developers might look down on doing this, but in my opinion, developers often need to test their own work more to experience the impact of cutting corners.
Create fewer bugs: This may sound flippant, but the easiest way to reduce debugging time is not to introduce bugs to begin with. More automated tests, such as with unit testing (see Chapter 12, “Agile Technology”) are often the best solution.
Buffering Between Stages
The time it takes to model a character can vary considerably. A simple character variation can take a day, whereas a boss character can take weeks. We don’t want this variation to cause times when the person who will rig the character has to wait around. Inserting a buffer between the two stages of character work allows a small backlog of models to exist so that a character is always waiting to be rigged. Care must be taken to limit the buffer’s WiP so we don’t impact the character cycle time too much.
Too much buffering can also slow down cycle time, so putting WiP limits on buffers is useful as well.
Buffers
A mailbox is an example of a useful time buffer. Mail delivery can vary by hours, and nobody wants to stand around waiting for the carrier to show up. That’s why we have mailboxes: They are a buffer between the carrier and recipient!
Kanban at My Local Coffee Shop
A few years ago, I pulled into one of the many local coffee shops, as I did every day. If, like me, you are a straight-coffee drinker who frequents coffee shops, you probably appreciate that you don’t have to wait for all the lattes, cappuccinos, and so on, ordered ahead of you.
The barista works on those exclusively, and the cashier can directly pour your coffee for you.
That day at my local shop, a light bulb turned on; I suddenly realized they were applying Kanban!
At a coffee shop, the empty coffee cups, marked with your (often misspelled) name and order details, are the Kanban cards. Based on how many there are between the cashier and barista, this informs them of what they should do next.
Looking at a hypothetical coffee shop Kanban board in Figure 6.4, my coffee goes from the “order” column to the “leave” column directly.
Figure 6.4 Coffee shop Kanban
Based on the size of the line, the barista and cashier might help one another out. If the line is long and the barista has nothing to do, he will ask people in line what drink they want and start making that before the cashier takes their order.
Conversely, if the line empties, but there is a backlog of unfinished drinks, the cashier will join the barista in making drinks.
This benefits everyone. A key metric for Starbucks is the customer cycle time: the amount of time it takes between walking in the door and when you walk out with your drink.
The critical path for coffee drinkers and latte drinkers isn’t the same, but it isn’t entirely separate; much as I personally would enjoy it, there is no separate cashier line for coffee drinkers. Starbucks has chosen not to optimize specifically for us straight-coffee drinkers for a good reason.
This is similar to the approach you might use for asset types. Although every asset will have a large variation of effort needed (like that between coffee and a latte) and partially separate paths, measuring every asset’s cycle time will still give us valuable information.
The goal isn’t to achieve a uniform cycle time for all assets, just as people who order lattes should expect to wait longer at Starbucks than us super-efficient coffee drinkers.
The Difference with Scrum
Besides the shift away from timeboxed Sprints and planning on-demand, very little needs to be different between Scrum and Kanban. The Scrum Master and Product Owner are still useful roles, and the meetings, such as Reviews, Retrospectives, and Daily Scrums, are still held.
Definition
Sometimes Kanban teams will refer to the “Daily Scrum” as the “Daily Standup.”
Often, teams exploring game behaviors in pre-production—exploring character behaviors and budgets—will start shifting into mass-producing more characters and will sense the need to abandon Sprint timeboxes and start measuring cycle times instead. This should be a smooth transition with very little change in roles and practices, except those introduced in this chapter.
When teams using Kanban return to using Sprints, they’ll often bring useful habits back, including applying WiP limits on their Sprint task boards and focusing on getting PBIs finished one at a time rather than near the end of the Sprint.
What Good Looks Like
When our level production teams introduced Kanban in 2007, they were able to reduce the cycle time of levels production by 56 percent over the course of several months by
Reducing handoffs with concept artists, as described earlier
Raising the priority of reducing export times with the tools support team
Reducing the batch sizes of levels by finishing portions of levels divided across streaming boundaries, reducing iteration times to increase the pace of improvements
Developing a better way of building levels (in Unreal rather than in Maya)
These improvements were driven by members of the team who were focused on reducing the cycle time of level production. It almost became a game with them to find ways to shave a few hours off every week.
Summary
This chapter covered the basics of Kanban as applied to areas of game development, such as content production and support (live and services). Later chapters dive into more detail for each application.
Additional Reading
Brechner E., and J. Waletzky. Agile Project Management with Kanban. Redmond, WA: Microsoft Press, 2015.
DeMarco, T., and T. Lister. 1999. Peopleware: Productive Projects and Teams, Second Edition. New York: Dorset House Publishing.
Katzenbach, J. R., and D. K. Smith. 2003. The Wisdom of Teams: Creating the High-Performance Organization. Cambridge, MA: Harvard Business School Press.
Keith C, Shonkwiler G. 2018. Gear Up!: 100+ ways to grow your studio culture, Second Edition.
Larman C., and B. Vodde. 2017. Large-Scale Scrum: More with LeSS. Boston, MA: Addison-Wesley.