Trouble with Audience Analyzer 2.5

Let’s check back in on Kate, Ben, and Mike. The last release worked out really well because the team had a good idea of what their users needed from the beginning. When they started thinking about what features will go into Audience Analyzer 2.5, some of the problems they set out to correct using Agile practices came back.

Ben: Is it so hard for the team to tell me what features will be in the next major release? We can’t just tell our customers to come to every demo and hope we’ll get around to doing their requests.

Kate: Look, I get it. We all get it. You’re asking for a really important feature, and everyone wants it yesterday.

Mike: The problem is that they’re all really important features.

Kate: Right. And everyone wants all of them yesterday.

Ben: Hold on, guys. I’m not the bad guy here, but we’ve got a business to run. This used to be a well-oiled machine. The first few iterations were great. We all knew what we needed to do, I could always tell the senior managers exactly what would be in the next release. But lately, it seems like one, then two, then three features don’t quite make it in. And worse, it seems like we keep finding bugs, and you guys never used to deliver low-quality work. What’s the deal?

Kate: Honestly, Ben, I don’t have a good answer. Something’s just, well, off about the way we plan our work. I can see us slowing down, but I don’t know what to do about it. I feel helpless here. We’ve got a backlog of work that we pull into each iteration, and that’s good because it means we can deal with uncertainty. It’s not that we’re uncomfortable doing long-term planning, but... well...

Mike: I see where you’re heading with this, Kate. We used to basically just be responsive: do this task now, then do this next task. Now everyone on the team has a good idea of what features we’ll be doing in two months, three months, even six months. It’s almost like that future work is so important that we feel like whatever we’re working on today is just blocking us from doing what we need to do tomorrow. We feel like we’re always late, and that’s causing us to make mistakes and cut corners. It feels like there’s all this work that hasn’t been done yet, so we have to get everything done as fast as possible. It’s really stressful, and it’s affecting our quality.

Ben: I know that our projects don’t have to be like this. I have no idea how to get there. Do you have any idea how we can improve things?

Lean is a mindset (not a methodology)

We’ve explored Scrum and XP, which each have a mindset component (values) and a method component (practices). Lean is different. It’s not a methodology, and it doesn’t include practices. Lean is a mindset based on principles that are all about making sure that the process your team is following is lined up to help you build products that are valuable to your customers. Where Scrum gives you a process to follow as a template (roles, planning meetings, sprints, sprint reviews, retrospectives), Lean asks you to take a look at the way you’re working today, figure out where you’re running into trouble, and apply Lean principles to correct those problems. Rather than telling you exactly what to do, Lean gives you the tools to see where the process you’re using is actually getting in the way of meeting your goals.

Lean teams look at the way they’re working today, and figure out where they are habitually running into problems. Then they make changes to improve the way they work.

Lean principles help you see things differently

Lean is divided into Lean principles and thinking tools that help you use those principles when you build software. All of the Agile methodologies are influenced by Lean thinking, so many of the ideas you’ll learn about in this chapter will be familiar to you or build on concepts you’ve learned in previous chapters. But Lean asks you to apply these concepts to the process you’re using to develop software and to constantly improve the way you work. Teams that use Lean even have a name for applying these principles and thinking tools: they call it Lean thinking.

More Lean principles

Remember the meeting of the minds at Snowbird we talked about back in Chapter 2? When they talked about the Agile principles of self-organization, simplicity, and continuous improvement, these last three Lean principles were an important part of that discussion.

A Lean mindset keeps your team focused on building the most valuable product possible in the time available.

Venn Magnets

You spent all night arranging magnets on your fridge into a really useful Venn diagram that shows which values and principles are specific to Scrum, which are specific to XP, which are specific to Lean, and which are shared by all... and then someone slammed the fridge door and all the magnets fell off. Can you put them back in the right places?

Venn Magnets Solution

Here are the magnets restored to the correct order. Lean and XP share a focus on empowering the team and all three of them focus on the last responsible moment and feedback loops. Lean and Scrum share an explicit focus on commitment.

That’s right. They are the things Lean teams do to support the decisions they make.

The whole point of Lean is to see where the team is doing work that is getting in the way of building a valuable product. If teams make use of the iterations and feedback thinking tool, they’ll be able to see the impact of the changes they’re making and use that impact to amplify learning. If they use the last responsible moment thinking tool, they’ll be able to decide as late as possible while building their product. That’s how the thinking tools and the principles are linked.

Some thinking tools you haven’t seen before

Now that you’re familiar with the Lean principles and some of the thinking tools that have been used in Scrum and XP, it’s time to take a look at the thinking tools specific to Lean. Lean teams use these tools to get a handle of the root causes of their process problems, and then fix them.

Seeing Waste

Before you can eliminate waste, you need to see it—but that’s easier said than done. Have you ever had a pile of clutter in your home that stubbornly stuck around for a while? After a few days, you don’t see it anymore. That’s exactly how waste in your process works, too, and it’s why seeing waste is an important Lean thinking tool.

Find the work your team is doing that isn’t helping you build a valuable product, and stop doing it. Is your team writing documents that nobody reads? Are you spending a ton of time manually doing work that could be automated? Putting a lot of effort into discussing and estimating features that probably won’t make it into the finished product?

Value Stream Mapping

This tool will help a Lean team find the waste in the process they’re using to build software. To build a value stream map, find the smallest “chunk” of the product that the customers are willing to prioritize on your backlog. Then think back through all of the steps the team took to build it, from when it was first discussed until it was delivered. Draw a box for every one of these steps, using arrows to connect the boxes. Next, track the amount of time it took you to perform each of the steps and the amount of time you waited in between each step. The time you spent waiting in between steps is waste. Draw a line that moves up to show that the project is working, and moves down to show that the project is waiting. Now you’ve got a visual representation of the work and the waste!

Queuing Theory

Queuing theory is the mathematical study of queues. Lean teams use queuing theory to make sure that people are not overloaded with work, so that they have time to do things the right way. The queue that needs to be optimized in software development could be a list of tasks, features, or to-do items for a team or for an individual developer. If you think of a backlog as an example of a queue, you’ll recognize that tasks that go into the backlog first are usually the first ones to be completed...unless someone, like a product owner, explicitly changes the order. Lean tells us that making a team’s queue of work public, and making it central to the decision-making process, helps teams to deliver more quickly. Queuing theory is often used by Lean teams to experiment with adding queues in a system to even out the flow of work through it.

Pull Systems

When a team organizes all of its work into a backlog and then has developers pick up a new task as they finish an old one, they’re using a pull system. The backlog is a queue of work. Instead of having users, managers, or product owners push tasks, features, or requests down to a team, they’ll add those requests to a queue that the team pulls from. Pull systems are about only working on one thing at a time and pulling work into the next phase of a process as a person frees up to work on it. That way people get to focus on doing the best work they can for each task they take up, and the product is built with attention to quality and efficiency.

Lean teams use value stream maps to find and eliminate waste.

More Lean thinking tools

The rest of the tools help teams keep their options open when they’re working, and constantly make sure that they are working on the most valuable thing.

Options Thinking

When a team is deciding which features will be included in an upcoming release, most people think of scoping as a commitment between the team and end users. Lean teams know that what the process is really doing is determining the options that the team might take in order to deliver value with each release. When we discussed Scrum, we talked about how a Scrum team doesn’t spend a lot of time modeling and tracking dependencies between tasks. Instead, the team is free to add and remove tasks on the task board every day during the Daily Scrum. These tasks are options, not commitments: the tasks don’t have due dates, and there’s no such thing as a “late” task that causes the rest of the project to blow a deadline. By freeing up the team to think about their work plans as options, they can make changes when they need to and make sure that they can do what’s best for the product instead of over-committing.

Cost of Delay

If a task is higher risk, it costs more to delay work on it than it would if it were low risk. Some features aren’t useful at all if they’re not completed within a certain time window. Understanding the cost of delaying each of the tasks in your team’s queue can help you make better decisions about which tasks must be completed first.

This is one reason that Lean teams develop a delivery cadence for releasing new features. This means that instead of committing to deliver a certain set of features on a specific schedule, the team commits to delivering the most value that they can at regular intervals.

Perceived Integrity/Conceptual Integrity

Lean teams are always looking to build integrity into their products from the very beginning. They divide their thinking about this Lean principle into perceived and conceptual integrity. Perceived integrity means thinking about how well a feature meets the needs of the user. Conceptual integrity means thinking about how well features work together to form a single unified product.

Set-Based Development

When teams practice set-based development, they spend time talking about their options, and change the way that they work in order to give themselves additional options in the future. The team will do extra work to pursue more than one option, trusting that the extra work will pay for itself by giving the team more information so that they can make a better decision later. This allows the team to gather more information about multiple options at once, and make the decision about which option to pursue at the last responsible moment.

Lean teams establish a delivery cadence in which they release the latest set of completed work at regular intervals.

Measurements

Just like Scrum focused on transparency, inspection, and adaptation, Lean teams measure the way their system is working before making changes and then measure the impact of the changes they do make.

• One measurement is cycle time, or the amount of time it takes for a feature or task to be completed from the time a developer begins to work on it until it’s delivered.

• Another measurement is lead time, or the amount of time it takes from when a feature is identified until it is delivered. Cycle time is often used to measure the effectiveness of changes to the development process, while lead time is used to measure changes to the requirements gathering and support processes as well.

• Teams will also measure flow efficiency, or the percentage of the total time for a feature that the team spent actually working on it (as opposed to waiting).

Cubicle Conversation

The people on the Audience Analyzer team know they’ve got a problem with their process. But they don’t agree about what’s causing it.

Ben: I’m fine with cutting down our scope. I’m happy to compromise. But the team really needs to get their estimates right.

Mike: ok...but it’s not always that simple. Remember that privacy option feature in the last release? I think it’s a pretty good example of why this isn’t so easy. First you wanted it to be really intrusive, so that all users had to set their desired level of privacy. Then you wanted it to be set by our customers. Then, just about three days before we were going release it, you decided that we should default it to a strict privacy profile and allow both of them to change the profile at will.

Ben: Well, our initial market analysis told us to do it one way, but then our subsequent research showed that we needed to change our approach. Making those changes made our last release a huge success. I know they were late in the game, but we needed to react to what the market was telling us.

Mike: I’m glad we made those changes too. But you have to see that it’s really hard to estimate how long it’s going to take to build a feature weeks in advance when requirements are always changing.

Ben: But you’re supposed to embrace change, right? Can’t you get better at estimating the work and tracking dependencies so we can make more informed decisions when those changes come up?

Categorizing waste can help you see it better

Lean tells us that we need to eliminate waste. But what, exactly, does that waste look like? How do we spot it? One thing that can be really helpful is to think of categories of waste in software development projects. Many Lean teams use value stream maps to see how much time is wasted in the development of the features they deliver. Once the team sees that there’s waste in their process, figuring out what type of waste it is can help them come up with improvements to eliminate it. Luckily, you don’t need to come up with these categories on your own. Lean practitioners have identified seven wastes of software development:

• Partially done work

If you start doing many different things at once and don’t finish the tasks you start, you’ll end up with a lot of work that’s not ready to be demonstrated or released at the end of your iteration. Partially done work happens all the time on software projects, whether or not the team is thinking about the costs of multitasking. Sometimes it just feels really productive to start something new while you’re waiting for information or approval on another task and that can lead to that first task being partially done when the timebox is complete.

• Extra processes

Extra processes are those that don’t actually help you deliver the software but add work to the team. Sometimes teams will put a lot of effort into documenting a feature that never gets delivered. Sometimes there are a never-ending series of status meetings that are meant to show the team that they’ve got management support, but which really just boil down to a lot of extra processes—like asking the team to create special status reports and gather information about each task in development.

• Extra features

Team members often get really excited about a new piece of technology. Sometimes they insist on including extra features that they think are brilliant but which no one asked for. It’s a common misconception that any kind of innovation on an existing idea is a benefit to a project. In truth, any new features that are added on by the team are taking time away from the features that the end users have asked for. That doesn’t mean that teams can never be the source of good ideas, but those ideas need to be vetted and presented as options before the team wastes effort building them out, instead of what was asked for.

• Task switching

It’s really common for managers to lose track of the number of requests they’ve made to a team, and just assume that there’s no cost to giving the team more work to do without adjusting anyone’s expectations. Couple that with the fact that software developers often want to overcommit in order to impress their bosses and teammates, and you see how people end up switching between three, four, five or more tasks that are all supposed to get done at the same time. That’s why task switching is an extremely useful concept in identifying software project waste. Any time a software developer needs to multitask between two competing priorities, time is wasted.

The seven wastes of software development help your team see the waste in your process, so that you can eliminate it.

• Waiting

Sometimes teams need to wait for someone to review a specification and they can’t get started until that’s done. Sometimes they’ll need to wait for infrastructure to set up physical hardware, or a database administrator to provision a database. There are many legitimate reasons that your team will need to wait during a software project. But all of that time is waste, and should be reduced wherever possible.

• Motion

When teams sit in many different locations, just the effort it takes to get from one person’s desk to another’s can add a lot of waste to your project. Motion is all that time wasted in transit while working.

• Defects

The later a defect is found in the process of building software, the more time that’s wasted finding and fixing it. It’s much better if defects are found as soon as they’re injected, by the developer who put it there, rather than by any testing process later in the development process. The more a team focuses on quality-driven development practices and shared code ownership, the less time they waste on fixing defects.

there are no Dumb Questions

Q: It sounds like Lean teams never tell people when they’re going to be done with any specific feature. Do they just decide what to do at the last moment and deliver as fast as they can? That would never fly where I work.

A: Teams that use a method that was developed with Lean thinking in mind—like Scrum or XP—focus on delivering small batches frequently. But rather than trying to figure out each and every task up front, they agree to build the most valuable thing they can build within the time frame. Lean teams know that just because customers say they need something at the beginning of the planning period, that doesn’t stay true the whole way through the project. So Lean teams see it as positive that they can change their priorities when the business needs it.

A traditional team might create a Gantt chart that seems to predict exactly which team members will work on each task, which are on the critical path, and when each milestone will occur. While that gives everybody in the organization a lot of confidence that the team has spent time planning, the plan is almost always inaccurate before the work even gets started.

A team that’s using Lean thinking tools will focus on getting the process right. If the process is right, the highest value work rises to the top of the work queue at the start of every increment. That helps the team to focus on eliminating waste in the way that they approach and build each feature and deliver it as quickly as possible. A Lean team sees planning as a way to identify options, and they’ll make the best choice from those options as new variables come into play.

Q: Oh, OK. So Lean teams develop in sprints, like a Scrum team?

A: Not necessarily. Lean teams establish a delivery cadence, which typically means setting a timebox for each delivery. It could be two weeks or two months—Lean teams deliver frequently and in small batches on predictable timetables. They plan their releases to coincide with those timeboxes. In Scrum, the sprints are a combination of planning and cadence. But many Kanban teams decouple the cadence from the planning: they’ll plan new work items, move them through the workflow, and release whatever happens to be in releasable state when the cadence is done. Some people refer to this as iterationless development because the timebox is not tightly coupled to the planning.

Q: So about those seven wastes. Do teams really build in extra processes and features? My team always feels like we don’t have time for anything extra.

A: Yes, they really do! Sometimes the teams that are under the most pressure are the ones that create the most wasteful work. It’s the projects that are under the tightest deadlines that often set up multiple status-gathering meetings a day, or create new practices for developers to log their time and scrutinize the number of hours spent on each activity. Those activities amount to waste that makes it harder to get the product delivered.

Options thinking helps you counter waste. The more complex a problem is, the less you know about it up front. That’s why Lean teams commit to objectives, but not to specific plans. They agree to the overall objective, and to deliver the most valuable thing they can toward that objective within the timeframe. Thinking about the commitment in this way gives both the team and the organization the freedom to focus on delivery and that usually means higher quality products, faster.

Value stream maps help you see waste

A value stream map is a simple diagram that helps you see exactly how much of your project time is wasted waiting and not working. Sometimes it helps to draw out the process your team is using on a time line to show where waste is slowing you down. A value stream map can make it painfully clear how much of your team’s time is spent on work that doesn’t result in value for your customers. Once your team can clearly see the waste, they can work together to figure out ways to reduce the time waiting.

Creating a value stream map of your most recent delivery is a great way to get your team thinking about how to improve their process.

Trying to do too many things at once

After taking a look at the Stat Mapper’s development time line, the team found a lot of time was getting wasted waiting for testing resources and environments. They investigated a little further, and realized that developers were starting up a new feature every time they finished the development and unit testing. That meant that each developer sometimes had four and five features at various places in the test and development pipeline. And since the design was highly coupled, features had to be released in batches that translated to test and deployment delays.

Once the team had the process mapped out as a value stream, it was easier for the whole team to come up with suggestions for how to spend more time working and less time waiting. When the team met as a group to take a look at the value stream map, they came up with a few easy-to-implement suggestions:

• Testers suggested automating their integration tests to cut down on time in test.

• Developers suggested simplifying the design of components so that they could be released independently.

• The operations team suggested automating the deployment pipeline so that it’s easier to release frequently.

Anatomy of an Option

Lean teams use options thinking to give them leeway to make decisions. They plan their projects just like any other team does—but the difference is the attitude each person on a Lean team has about those plans. They see all of the work they’ve planned to do as options, not commitments. The team commits to an objective, but not a plan. This allows them to focus on meeting the objective and change the plan if a better way of accomplishing the goal presents itself. By not committing to each step in the plan, the team leaves themselves free to change the plan when new information presents itself.

Here’s how you might use options thinking on a project:

1. Define your objective and commit to that.

   Goal: Release an Audience Analyzer enhancement that will store data from three new sources and serve that data to our Stat Mapper application.

2. Define the tasks needed to accomplish that objective and consider the completion of those tasks one option for achieving the goal.

   

   When the team did their planning, they had an idea of how they wanted to do the work. But they thought of this as just one option, and left themselves freedom to choose another path.

3. Start working!

   

4. Change the plan as needed.

   

Options thinking lets Lean teams give themselves the freedom to decide as late as possible and reduce the cost of changes when they occur.

On a traditional team, a change in technical approach can be really disruptive. If you’ve committed to a detailed schedule, you might have set aside time and resources—say, development and DBA team members to work on a task that has programming and database work. You might have created milestones that you report status on to upper management for when the task was complete, or maybe you made this task a predecessor to work being done in the application, for example. That’s a lot of up-front planning.

What if one of your team members discovers that a solution already exists? That could cause your whole schedule to be recalculated, and resources would need to be redistributed. That kind of thing happens all the time! By treating the team’s original plan as an option, you don’t have a number of tasks and resource assignments that are dependent on that task to re-assign and re-orient.

Systems thinking helps Lean teams see the whole

Every team has a way of working. It might seem like you make it up as you’re going along, but there are always rules that everybody on the team follows (even if you don’t realize you’re following them). That’s what the Lean principle is about. It’s called see the whole, and it starts with recognizing that each person’s work is part of a bigger system.

When a team sees itself as a group of individuals with different functions, they tend to focus only on the improvements that can be made to their job. A programmer might focus on improvements that will make programming easier, a tester will focus on improvements to testing, a project manager might focus on improvements to scheduling or reporting on status. But when all of the team members see how their jobs contribute to a larger system, they can all start coming up with improvements that help the team reach its objectives together instead of optimizing one role above the others.

When everyone sees the whole, the whole team gets better together.

Say for example, a project is running late, so a project manager optimizes his job function by asking for written status twice a day from every member of the team. That will probably make the project manager’s job easier, because he always knows where the team is on every task. But it will also slow the team down and make it harder for them to get work done on time. Or, think of a programmer who believes that she can get so much more code written if she doesn’t have to write unit tests. That programmer might produce a lot more code, but the cost of finding and fixing defects in it will probably outweigh her added productivity.

Lean teams work to remove local optimizations like those and optimize the system together. They look at the system together and remove any activities that are getting in the way of building software fast and with high quality. Lean teams know that limiting their work to the micro-problem of how to make each team member more productive actually gets in the way of operating as a team.

Some “improvements” didn’t work out

The team took a hard look at their value stream map and realized that they needed to focus on finishing each task before they start a new one. Next, they started to think of ways to improve their coordination with the Operations team. Instead of trying to see the whole system, they focused on just the work that they could control.

Lean teams use pull systems to make sure they’re always working on the most valuable tasks

Traditional project teams push work through their systems. They attempt to plan all of the work up front and then control changes to that plan throughout the project’s execution. By predicting exactly how much work will occur and who will need to do it, traditional projects attempt to maintain the right resource mix through meticulous planning.

Teams with a Lean mindset work in the opposite way: they use pull systems. In a pull system, each step in the process is triggered by the later step, pulling the output of that previous step only when it’s completed. By having the later step in the process pull tasks from the former, Lean teams finish each feature as quickly as possible. Pulling work through the system establishes a constant flow of finished work through it and doesn’t overload it with local optimizations.

there are no Dumb Questions

Q: Options thinking seems weird. Isn’t it better to have a really accurate picture of what will happen on a project, and to be as predictable as possible?

A: Think about the last time you started a project. How much did you know about how it was actually going to play out? If it was like most projects, there were surprises from the time you started right up until you were done. Lean teams recognize this—rather than try to predict exactly what’s going to happen and then measure against that prediction, they treat even the broad strokes of the project as optional. That doesn’t mean they won’t happen, but if a better option comes along, they’ll take it.

Q: That sounds pretty theoretical. What does that mean in practice?

A: Options thinking in practice can mean starting down multiple paths at the same time to solve a tough technical problem that doesn’t have a clear answer. It can mean being open to changing scope and strategy in order to meet an objective and deliver the most value. Lean teams keep their options open by focusing on the business outcome that’s needed. They don’t plan out every detail of the implementation of each task it will take to accomplish their goal and try to stick to that plan.

You’ve seen this idea before. Another name for options thinking is making decisions at the last responsible moment—just like you learned about in Chapter 3. And that’s another example of how the Agile Manifesto signers had Lean thinking tools in mind when they wrote the agile principles.

Q: I have enough to worry about just developing my projects. Now I’m supposed to think about the whole system I’m working in, too?

A: A lot of people feel that way. But focusing just on the job you’re doing can lead you to work in a way that makes it harder for your team to actually get work done. While it might seem to you like you’re getting a lot more done when you make an improvement that’s focused just on your role, it’s more important that the whole process you’re working in is producing high-value software than it is that you are able to do more in your functional silo.

While it might seem like each person focusing on doing their own jobs as well as they can will get a product out the door sooner, that’s rarely true. The act of building software most often depends on a lot of people collaborating with each other to figure out what should be built, develop it, make sure it works, and make it available to the people who need it. Focusing too much on any of those steps can lead to problems in the software itself. Lean teams solve this problem by asking each person on the team to think about the whole system and not just the work they are personally responsible for. That’s where the Lean thinking tools from earlier in the chapter can help—like seeing waste, using options thinking, taking measurements, and figuring out the cost of delay for each feature.

Q: I get some of the other thinking tools, but I’m still not clear on cost of delay. How do you figure that out?

A: For some features it can be pretty obvious. If you’re building software to implement a new rule in a tax code, then the cost of missing tax day is probably pretty big. For other features, however, it can be a lot harder to understand. That’s why Lean teams focus on trying to figure out not just the priority of the features they’re working on, but each feature’s cost of delay too. Making sure that the team talks about the cost of delay can make sure that teams are working on the highest-value feature at any given time.

One way to figure out cost of delay is to ask about it. When a product owner is describing the feature to the team in sprint planning, or when the team is committing to what will be in an upcoming release, have a discussion with the person who prioritized the features to make sure the team understands the cost of delaying that feature.

Sometimes asking questions about it is enough to cause the product owner to re-think the current priorities. Some teams use heuristics and practices like assigning a business value or cost of delay number to each of the features being planned in a sprint planning session so that it’s easier for teams to understand the cost of not releasing it quickly.

The most important thing to remember here though is that you should consider the cost of delay when you’re determining the priority order of work in an increment.

Q: What does that “queuing theory” stuff have to do with anything?

A: Once you see your software process as a system, it’s not hard to start thinking of all of the work that goes through it as part of a queue. If you think of each feature as queued to go from the first step to the second, and then the third, and so on, you’ll start to see how queuing theory applies to the work your team is doing. Have you ever noticed how some supermarket checkout lines move more quickly than others? The same principles that govern why some lines are faster or slower also explain why your team builds features quickly or slowly.

Lean teams think about their process as one big (mostly) straight line, and then set about trying to find the most direct route from identifying a feature to be delivered to releasing that feature with as little waste in between as possible.

Mike: We’re setting aside a little time in each increment to make headway on the test automation, and we’ve started refactoring the components so we can release them independently. Everything is popping into place. In a few months we’ll be cruising at twice the speed we are now.

Kate: Wait, what? A few months?! When we started tracking our velocity I thought we were giving everyone visibility into how much we can do. Now the whole company is obsessed with it. Whenever our velocity number goes down, I have to go to meetings with senior managers and explain why.

Mike: But... but that’s not how velocity and story points work!

Kate: Tell that to the steering committee. They’ve asked us to come up with story point estimates for features earlier and earlier, and set schedules way out into the future based on them.

Mike: Ugh, that’s so frustrating! The steering committee needs to understand that we’re sharing our velocity with them so they can help us make the right choices, not so they can use it to push more work onto us. How is this any different from when we used to plan big releases with big specs?

Question Clinic: Least worst option

Kanban uses a pull system to make your process better

Kanban is a method for improving your process. It’s based on the Lean mindset, the same way that XP and Scrum are based on their specific values. But it’s a little bit different from XP or Scrum, in that it doesn’t prescribe roles or specific project management or development practices that tell you how to work on a team. Instead, Kanban is all about taking a look at the way you work today, figuring out how work is flowing through your system, and then experimenting with small changes and WIP limits to help your team establish a pull system and eliminate waste.

Use Kanban boards to visualize the workflow

A Kanban board is a tool that Lean/Kanban teams use to visualize the workflow. It consists of a board—often a whiteboard—divided into columns, with cards in each column to represent work items flowing through the process.

A Kanban board looks a lot like a task board, but they’re not the same thing. You’ve seen task boards in our discussions of Scrum and XP, so it’s easy to look at a Kanban board and assume it’s basically the same thing. It’s not. The purpose of a task board is to make the state of current tasks clear to everybody on a team. Task boards help a team stay on top of the current status of their project. Kanban boards are a little different. They are created to help a team understand how work flows through their process. Because work items are kept at a feature level on a Kanban board, they aren’t the best way to know exactly which task each team member is working on—but they’re great for helping you see how much work is in progress in each state of your process.

How to use Kanban to improve your process

The core practices of Kanban are a series of steps that teams execute in order to see how their system is working, and then create a pull system that moves work efficiently through their workflow.

The team creates a workflow

The first step in improving a workflow is to visualize it, and that’s what’s going on here. The Audience Analyzer team got together to talk through the set of steps the team follows every time they build a new feature. There are exceptions, of course: Sometimes the team never schedules a feature to be delivered. Sometimes the product manager asks for it to be built as an urgent priority because a user needs it right now, and everybody drops what they’re doing to make it happen. Or bugs might get found in test too late, and they don’t get fixed. Even though the process isn’t always followed exactly in this order, the team tried to create a picture of phases a feature is supposed to go through when the team builds and delivers it.

After spending a while discussing those exceptions (and a few more) the team agreed that most often, they follow a workflow that looks something like this:

Next, they mapped out their process on a Kanban board so they could see which features were in which workflow state. They drew out columns on a task board that match the states they’d defined and then figured out which state each of their current features were in. For each feature, they created a sticky note and put it in the correct column on the board.

When they created their Kanban board, they decided not to put a Plan column on the board. The team always held a two-hour meeting at the beginning of each delivery increment to plan out the work they’d do, so features would never be in the “Plan” state for longer than two hours. It just didn’t seem useful to track that meeting as a state.

Kanban is not a project management methodology. It’s a method for process improvement that works by visualizing your team’s actual, real-world process.

One of the biggest misconceptions about Kanban is that it’s basically just Scrum without sprints. It’s not. Kanban is not intended to be a project management methodology. You use a Kanban board differently than you use a task board: a task board is for managing a project, while a Kanban board is for understanding your process.

Remember how you used a value stream map to visualize the actual path that a real feature took through a project? A Kanban board does the same thing, except instead of just following a single feature, it follows all of the work. Once you and your team can see the whole workflow visualized on the board with work items moving through it, you make adjustments to it in order to make it as accurate a picture as possible.

The goal of Kanban is to help the team inspect its own way of working, so that they can make changes collaboratively in order to release small increments as frequently as possible. Kanban doesn’t tell teams exactly how long their timeboxes should be, or what roles should be on their teams, or what meetings they should have on a frequent basis. It helps the team figure those things out for themselves.

So how does the Kanban board fit in? Every team works a little differently, and those differences are reflected in the columns on the Kanban board. If your team always spends a few weeks building a proof of concept for each new feature and does a demo with a group of users before a project really gets started, then you’ll add a column to your Kanban board that represents that state. And as you work, you’ll discover new columns and add them, so that the whole team gets a clearer and clearer picture of how they work.

Cubicle Conversation

The team spent a few increments just watching how features moved through their process. As they observed, they learned a lot about how they were working.

there are no Dumb Questions

Q: I’ve heard about “Lean/Kanban” before, and I see how Lean and Kanban work. But how do they fit together?

A: Lean is the mindset behind Kanban. Kanban relies on Lean, just like Scrum and XP rely on their values. Kanban builds on the thinking tools in Lean, and Lean teams use systems thinking to improve their process.

If you’re working on a team that’s been building software for a while, it’s likely that you and the rest of your teammates are following an unwritten set of processes and policies to do your work. Because those rules and processes are not explicit, small misunderstandings can evolve over time that end up making the team less deliberate than they could be in the choices that they make during development.

Kanban helps teams to think about the way they’re working, and to really examine the decisions that they they make when they’re building a product. By asking the team to visualize the system together and then measure how work flows through it, the team can see the impact of the way that they work on the amount that they get done—and on the time it takes to do it.

By creating a visual map of their process, they can see the effect of any changes they might make together. Kanban’s practices of visualizing the workflow, experimenting with WIP limits, and managing flow effectively get everyone on the team to start using the Lean thinking tools—like see the whole, cost of delay, pull systems, and queuing theory—collaboratively.

Q: Earlier you said that Kanban means “signal card.” What does that mean? Are those the same as stickies?

A: Good question. In manufacturing, a Kanban is a physical token—like a card with a part number written on it—and it’s the basis for a pull system in that environment. In the Toyota Production System, when the team is running low on a specific part, they put a kanban card (literally “signboard” in Japanese, ) with its part number into a bin. A supply team exchanges the kanban for more parts from a central supply. The team uses cards to pull parts as needed.

For a software team, Kanban uses work items on a board to implement a pull system. And the same ideas still apply, even though software teams are doing creative intellectual work and not assembly line manufacturing. Just like TPS uses Kanbans to cut down on wasteful processes and excess inventory in building cars, software teams using Kanban to improve their software development processes cut down on wasteful work using the same Lean principles.

Q: I’m starting to get it, but something doesn’t make sense. Can you explain again how limiting the work you do gets more work done?

A: When people are focused on a lot of different goals and have their own ideas about how to work, they can end up working really hard toward different ends. When that happens, everybody seems busy, and they feel like they’re doing the most they can to get a product released. But the truth is, until they are all focused on the same goal, they might actually be slowing the release of a product down.

By limiting the number of work items “in flight,” Kanban asks teams to think about the end goal and eliminate any work they’re doing that isn’t related to it. Since Kanban teams measure the lead time of a feature before and after limiting WIP, they always know whether an experiment with WIP limits is slowing things down or delivering features faster. By focusing on delivering small increments as fast as possible, Kanban helps teams remove the waste in their process, and build software faster and with higher quality than they could before.

The team is delivering faster

It took a couple of tries, but after some experimentation (and a lot of lead time measurements) everyone could see real improvement. The first time they set a limit, it ended up slowing the team down a bit. So everyone got together and had a great discussion, and they decided to try making the number a little higher for the next increment. That seemed to do the trick. Once the team had that WIP limit in place, they found that they were starting and completing more features in each increment than ever before. Even better, they all started helping each other when they ran into problems. Pretty soon it felt like the work was more under control than it had ever been. Ben was especially happy, because the team was much more predictable than they had been in the past.

Cumulative flow diagrams help you manage flow

Kanban teams use cumulative flow diagrams (or CFDs) to find out where they are systematically adding waste and interrupting their flow. They chart the counts of work items in each state over time, and use that to look for patterns that might be affecting the team’s throughput (the rate that they can complete work items). CFDs give the team a visual way to keep track of how the system is working.

Once the team gets used to reading CFDs, they can get a good idea of the impact of process and policy changes a lot more quickly. Teams are always looking to have a stable development process with predictable throughput. Once a team has identified the right WIP limits to establish pull, they can start to look at how their working agreements and policies affect the way they work as well. If a team makes it a habit to constantly review their CFD, they all get a sense of how their suggestions and changes affect the amount of work the team can get done.

Note

This is just a broad overview of CFDs—we wanted to give you enough exposure to CFDs so that you can get a sense of how your process is working. Once you’ve been using Lean and Kanban for a while, you’ll want to know more about common patterns in CFDs and how to interpret them. CFDs can be a really valuable tool, and they’re actually pretty easy to build. We give you a step-by-step guide to building CFDs and using them to improve your workflow in our book, Learning Agile.

Kanban teams talk about their policies

One you’ve got the steps in your process written down and you’ve started looking at how features flow through it, you’re ready to think about all of the rules team members are following when they do their day-to-day work. It’s pretty common for people to come up with their own rules as they’re working that guide their behavior and the decisions they make. Many of the misunderstandings and miscommunications that your team will run into while they’re working come from these unwritten rules. But when you really start talking about the polices, and people on your team are having a truly open and collaborative discussion, your whole team can begin to work together to avoid many misunderstandings up front. Teams use these policy discussions to come up with working agreements together that make team interactions smoother and keep everyone focused on changing policies rather than fighting with each other when things go wrong.

Here’s the working agreement the Audience Analyzer team came up with when they decided to make their policies explicit:

Talking about the policies your team is following and writing them down helps you get everybody on the same page.

Feedback loops show you how it’s working

Kanban teams are really focused on understanding the improvements they make to their process, so they explicitly create feedback loops to measure the impact of every change they make. They do this by taking measurements, then using the data from those measurements to make changes to the way they work. As they change their process, their measurements change, which they use to make more changes to their process, and over and over and over...

Teams use their feedback loops to establish a culture of continuous improvement, making sure that everyone helps to take measurements and suggest changes. When everyone is involved in measuring, changing, and repeating, the whole team starts to see each process change as its own experiment.

Now the whole team is collaborating on finding better ways to work!

Now that the CFD and lead time are shared with the whole team, they’re coming up with suggestions to make things work better every couple of weeks. Not all of them work, but that’s OK, because the team learns together from every experiment they try. They’ve dramatically improved their lead time, and everyone feels engaged and in control.

Now that the team is improving collaboratively, they have more control and they’re getting more done!

Exam Questions

1. Value stream maps are used for all of the following except:

   1. Understanding a feature’s lead time

   2. Finding waste in a process

   3. Discovering new features to build

   4. Understanding a feature’s cycle time

2. Sean is a developer on a team that’s building financial software. His team has been asked to build a new trading system. He and his team had a meeting to come up with a picture of the workflow they’re using. Then they put the process on a whiteboard with columns for each step in the process. After a few weeks of watching the work items the team was working on progress through the columns on the board, the team noticed that there were a couple steps in the process that seemed to get overloaded.

   What’s the BEST thing for the team do next?

   1. Work with the team to get better at doing the work in the steps where work is slowing down

   2. Add more people to the steps that are slower

   3. Focus on finishing the work on the board

   4. Limit the amount of work in progress allowed in the steps that are overloaded

3. Lean is different than methodologies like Scrum and XP because it is a ________________ with ____________________

   1. Mindset, thinking tools

   2. Methodology, practices

   3. Process improvement plan, measurements

   4. School of thought, principles

4. A Lean team looked at all of the work items in their process and paid attention to how they were progressing between each stage in their process. They then focused on how to keep the line of work moving at an even rate. What thinking tool helps them see work as a line of features being removed from the system when they are done?

   1. Seeing waste

   2. Last responsible moment

   3. Queuing theory

   4. Measurements

5. Which of the following BEST describes the Lean thinking tool “Cost of Delay”?

   1. Ranking features based on how soon your client needs them

   2. Assigning a dollar value to the time you spend building a product

   3. Understanding the time-criticality of each of the tasks in your team’s queue so that you make better decisions about which tasks must be completed first

   4. Understanding how much money you are losing when you delay your project

6. What are the seven types of waste in software development?

   1. Partially Done Work, Extra Processes, Task Switching, Heroics, Over-commitment, Defects, and Extra Features

   2. Partially Done Work, Extra Processes, Extra Features, Task Switching, Communication, Waiting, and Defects

   3. Partially Done Work, Extra Processes, Task Switching, Waiting, Motion, Defects, and Extra Features

   4. Partially Done Work, Extra Processes, Task Switching, Detailed Planning, Motion, Defects, and Extra Features

7. Which of the following BEST describes Kanban’s core practices:

   1. Visualize Workflow, Create Kanban Board, Limit WIP, Manage Flow, Make Process Policies Explicit, Implement Feedback Loops, Improve Collaboratively, Evolve Experimentally

   2. Plan Do Check Act

   3. Visualize Workflow, Observe Flow, Limit Work In Progress, Change Process, Measure Result

   4. Visualize Workflow, Limit WIP, Manage Flow, Make Process Policies Explicit, Implement Feedback Loops, Improve Collaboratively, Evolve Experimentally

8. Which of the following is NOT a Lean principle?

   1. Eliminate Waste

   2. Implement Feedback Loops

   3. Decide as Late as Possible

   4. See the Whole

9. Which of the following BEST describes how a Kanban board is used?

   1. To observe how features flow through a process so that teams can determine how to limit WIP and identify the most even flow of work through the steps in a workflow

   2. To track WIP limits and current task status so that a team knows how much work they have left to do

   3. To track defects and issues and create the fastest path for resolving problems in a product

   4. To help a team self-organize and see where bottlenecks are in their workflow

10. What is the lead time and cycle time for this feature based on the value stream map below?

    

    1. Lead: 22 days, Cycle: 30 days

    2. Lead: 30 days, Cycle: 22 days

    3. Lead: 52 days, Cycle: 30 days

    4. Lead: 70 days, Cycle: 42 days

11. When a team has established WIP limits and started managing flow, what’s the NEXT step in applying Kanban to their workflow?

    1. Implement feedback loops

    2. Make process policies explicit

    3. Improve collaboratively

    4. Deliver as fast as possible

12. Members of your co-located team are having a hard time meeting their commitments. They keep committing to more and more work each increment, but they’re not accomplishing their goals. Now there are many features in the first two columns of your Kanban board and very few in the later columns. Which of the following kinds of waste is NOT likely the cause of this team’s problem with meeting commitments?

    1. Task Switching

    2. Defects

    3. Motion

    4. Partially Complete Work

13. Which of the following is NOT a principle that is shared between Lean and Scrum?

    1. Last responsible moment

    2. Iterations and feedback

    3. Self-determination and motivation

    4. Eliminate waste

14. When the later step in a process moves work from the step before it, this is referred to as...

    1. Queuing Theory

    2. Waste

    3. A pull system

    4. Inventory

15. Which of the following is NOT an example of options thinking?

    1. A team tries two parallel approaches to developing a high risk feature that they have little experience with

    2. A team identifies goals that can be delivered early in a project to validate the design approach

    3. A team sets a hard deadline and list of dependencies to deliver a feature they’re unsure about

    4. A team spends time sharing knowledge together so that more people can work on all of the tasks that are planned for a sprint

Exam Answers

1. Answer: C

   Teams use value stream maps to find waste in their process. Value stream maps can give us valuable information about lead time and cycle time, but rarely provide insight into future product needs.

2. Answer: D

   This question is describing the initial steps a team takes when they are doing Kanban. The first step is to visualize the workflow, the second is to limit WIP.

3. Answer: A

   Lean is a mindset with thinking tools to help you think about problems in a way that identifies and eliminates waste. XP and Scrum are methodologies with practices that help you deliver software while adhering to Agile principles.

   Note

   Answer D is a pretty good answer, but answer A is definitely more accurate.

4. Answer: C

   Lean teams use Queuing Theory to examine the order in which work is done in their system. Then they try to find waste that is causing the team to spend time waiting when they could be making progress and removing completed work from the line.

5. Answer: C

   Cost of Delay is one of the criteria you can use to determine the order in which features should be developed. Understanding a feature’s cost of delay means thinking about the risk involved in it as well as demand, and missed opportunities you’re encountering because the team is working on the current feature.

6. Answer: C

   Lean’s seven wastes of software development (Partially Done Work, Extra Processes, Task Switching, Waiting, Motion, Defects, and Extra Features) are derived from the Toyota Production System’s seven wastes of manufacturing (Transportation, Waiting, Motion, Defects, Over-Production, and Extra Processing). Both lists are a useful way to categorize the common types of behavior that slow down production when building products and features.

7. Answer: D

   Only answer D includes all of the core practices of Kanban.

8. Answer: B

   “Implement Feedback Loops” is a core practice of Kanban, but it’s not a principle of Lean. The principles of Lean are: Eliminate Waste, Amplify Learning, Decide as Late as Possible, Deliver as Fast as Possible, Empower the Team, Build Integrity In, and See the Whole.

9. Answer: A

   Kanban boards are used to track how features progress through your workflow. When teams implement Kanban, they do it to understand how their process is working, not to track progress on a specific project. They are not used for tracking tasks, task boards are used for that.

   Note

   This is what people mean when they say that Kanban is a method for process improvement, and not a methodology for project management.

10. Answer: C

    Lead time is the total time the feature is in the system. In this case adding up all of the numbers on the Value Stream Map gives 52 days. Cycle time is the total time spent working, that’s 30 days.

11. Answer: B

    The order of the practices in Kanban is: visualize workflow, limit WIP, manage flow, make process policies explicit, implement feedback loops, improve collaboratively, evolve experimentally.

12. Answer: C

    The question mentioned that the team was co-located, so it’s least likely that time in transit is causing the team’s problem delivering. On the other hand, if they are constantly taking on new work without finishing the work they already have, it’s likely they will run into problems with task switching, partially complete work, and defects.

13. Answer: D

    The concepts of deciding as late as possible, releasing in short iterations in order to get frequent feedback, and self-organizing are common to both Scrum and Lean. Although Lean and Scrum are compatible with one another, Lean is focused on eliminating waste, while that’s not something that Scrum emphasizes.

14. Answer: C

    In a pull system, the later step in a process pulls work from the previous one. This keeps the workload even and is the least wasteful way to get work from the beginning to the end of a process.

15. Answer: C

    By setting a hard deadline and determining which tasks will depend on which, the team is locking themselves into an approach and cutting down the options they have to achieve their goal.