I often start my agile project management classes by asking students what they think of when they read or hear “agile” (I always think of a cheetah). Students typically respond with “fast, nimble, flexible, responsive.” I mention the cheetah in the context of chasing a gazelle: The cheetah’s goal is the gazelle, which is constantly moving and changing direction – much like the requirements and objectives of some projects. The cheetah must process the input of the gazelle’s changes in direction along with the changes in the landscape and respond to these changes in order to accomplish the goal of catching the gazelle.
Agile project management describes a set of methods and techniques that help project teams and organizations do the same thing: go fast and be responsive to change. In practice, agile methods are often blended with plan-driven approaches and other agile methods into hybrid approaches. These hybrid approaches are adapted to fit the culture of the organization, its structure, and its appetite for risk and change.
Agile methods are iterative, adaptive, and interactive and place a premium on teams and teamwork. In agile, teams are self-managed while accountable for delivering value regularly and frequently. Most agile frameworks use short, time-boxed iterations of focused work, often called sprints, during which constant communication within the team and with stakeholders ensures the team works on the right things toward the right outcomes.
In practice, most organizations blend plan-driven approaches with agile practices to create their own hybrid approach – the focus of this book.
Agile Life Cycle Models
Hybrid agile life cycle (Thomas, 2008)
Thomas’ model shows how each time box (iteration) has some planning work, development work, and a review (plan-do-check-act). The work is planned, the team commits to the work, the team performs the work, and the team reviews its work when the cycle is completed. The accumulation of completed work through iterations comprises a release. When the project is deemed complete, there are project closure and post-project close activities which are necessary to bring any project to an orderly close.
Scrum – Basic Practices
Because agile project management derives most of its practices and vocabulary from Scrum and, in fact, is often (right or wrong, for better or worse) discussed interchangeably with Scrum, it is critical to understand the core practices of Scrum as the basis for the practical application of agile.
Scrum can be very simple. At its essence, Scrum uses three roles, three processes, and three artifacts as the framework for organizing and performing work. The three roles are the scrum master, the team, and the product owner. The processes are sprint planning, the daily stand-up meeting or daily scrum, and the sprint review, which includes a demo of what the team did along with a retrospective of their performance and how to improve (remember – PDCA). The three artifacts are the product or project backlog, the sprint backlog, and a visual tool such as a burndown chart, burn-up chart, or a scrum board, used by the team to see their progress during a sprint.
The team plans a sprint, commits to completing specific work, and then works for a specified period of time with total focus to complete that work. The scrum master leads the team in scrum practices and helps them self-organize. At the end of the sprint, the team shows the product owner what they’ve completed. The product owner reviews and accepts or rejects the work. The team discusses what went well, not so well, and what to keep doing or change during the next sprint. The team takes this right into their next sprint planning session, which ensures they can iteratively and continuously improve.
Scrum life cycle
Release planning involves the selection of product backlog items that will be included in a particular release. Sprints will focus on a portion of that backlog for each sprint. Sprint planning focuses specifically on the work and tasks associated with each item on the sprint backlog, and the team finalizes their commitments while making specific estimates of what work will be accomplished in that sprint.
The sprint is where the magic happens. Each day, there’s the scrum – the daily stand-up meeting during which the team discusses what they did, what they will do, and any obstacles preventing them from accomplishing those commitments. At the end of each sprint, there is a demonstration, where the team shows what it has completed to the product owner and other stakeholders. The product owner potentially accepts it all as complete or accepts some of the work as complete while rejecting other work items.
The team then performs its retrospective – it examines how it performed in the previous iteration, and what the team must do to improve its performance. During each ongoing sprint, the product owner and the scrum master “groom” the backlog – this refers to the ongoing process of refining and prioritization of items on the backlog to assure that the backlog items are progressively better and fully understood and that what is being done provides maximum business value. This process is repeated until the release or the overall project is complete.
Agile Practices
The next few sections of this chapter will take a practical look at various practices such as sprint planning, estimating, user story development, release planning, and other elements of agile project management. We will examine how they are used on their own as well as how they work and fit within a hybrid approach to project management where phase-based and agile practices are combined for the best outcome.
Backlog Grooming
Organizations and teams using agile frameworks and practices to perform projects should follow certain processes in preparation for each iteration or sprint (assuming they are using these defined time boxes – not all agile frameworks do). The product backlog, the list of prioritized features and work items, needs to be constantly reviewed and updated. Many call this process, particularly within Scrum, the grooming of the product backlog.
In practice backlog grooming requires constant attention. It can be hard for those new to agile to understand that this practice never stops – stakeholders and team members must invest some hours in backlog grooming every week to ensure that each iteration or feature planning meeting is worthwhile. As a practitioner I learned how important this is – first at Promega, where we got a good cadence for this going, and then at CloudCraze, where getting a consistent backlog grooming cadence in place was initially a struggle.
Prioritization – Ensuring that highest-value items are at the top of the list, along with any mandatory dependency items
Discussion – Ensuring that the product owner and the team develop and evolve a shared understanding of the work and features
Sizing – Ongoing relative sizing of backlog items so that the team and the organization have an idea of how much work remains in the backlog, so as to inform other project and operational processes and decisions
As noted previously, this process must be constant and ongoing. If this breaks down or stops for any reason, the team runs the risk of attempting a sprint planning meeting without a groomed backlog of well-defined and understood stories. This in turn creates the likelihood that this work will be done in sprint planning, which diminishes its value, and also results in an inefficient sprint planning meeting. I encountered this problem in my early days with CloudCraze Software. A group of us came to a sprint planning meeting – I had created the expectation that the product owner would have reviewed the backlog and would propose a set of prioritized features for the nascent product development team to work on. The product owner had in fact not done that work, so we spent two hours doing it and then had to plan the sprint in a subsequent meeting. This resulted in the launch of the new sprint being delayed by a couple of days. These were days which, as a fast-moving software startup, we did not have the luxury to waste.
A constantly groomed backlog also means that the agile team always has a list of prioritized work that they can pull into an active sprint in case that sprint is going better than planned. Rather than stop and have to ask about it, the always-groomed backlog means the team can pull forward work and perhaps have a sprint that produces more output than originally planned.
Agile Estimating
Estimation of the product backlog enables a project team to roughly size how much work they are taking on or have left in a project or release. Estimation of the sprint backlog is critical to ensure that a team can make a realistic commitment for work to complete that is based on their capacity and their sizing and estimation of the work they are about to take on. Estimating in agile has proven to me to be one of the more challenging concepts to convey to people new to agile, especially if they come from a plan-driven project management background.
Estimating in agile projects requires a change in thinking. Agile estimating gets us talking about sizing the work, thinking about estimating in different ways, and using tools like analogous estimates to help us get a sense of what it’s going to take in order to deliver the project, to deliver value through the execution of this project.
Let’s consider how the plan-driven estimate might rely on processes and artifacts such as a work breakdown structure and decomposing the projects down to very fine level of detail both the deliverables and the tasks. Typically, plan-driven waterfall breaks a feature or deliverable down into a smallest logical component. Another thing that we typically do in plan-driven project estimation, and in fact is considered a best practice in plan-driven estimation, is that the person who will do the work, or somebody who is expert in the work, will give the estimate. We rely on a specialist or a discrete individual to do estimating of their particular deliverables and tasks.
Yet another assumption is that the entire project will be estimated upfront, along with a scope definition in order to lock this in. Then, we manage change very rigorously throughout the project life cycle. Again, the assumption is that the estimates are “one and done” upfront. There’s the assumption that we’re going to use estimates of effort and of the availability of resources to calculate the duration of any task and that we estimate and lock all of the features and functionality in at the start of a project.
Agile changes this estimating paradigm. For starters, estimates are happening at a higher level – they’re usually being made using a “this seems larger or smaller than that” approach, and not going through the work breakdown structure effort to try to get to that level of granularity. We depart from the model of relying on a subject matter expert or the person who will do a particular task, and instead we look at estimating as a team exercise on the assumption that in many cases agile teams are going to self-assign work, so at the outset we don’t know who may be doing that particular task. Therefore, the team needs to estimate so we have everyone’s input.
Rather than locking estimates in at the start of a project, we update estimates regularly, and we assume there will be change based on learning. One of the inputs to this is performance or team throughput. As we move through the agile project life cycle, we are constantly assessing the team’s capability and performance so that we have a sense of what can be accomplished within a particular timeframe, or conversely, what timeframe will be required to deliver all the functionality that the product owner may require.
Agile estimating focuses on a couple of critical things. It relies on the evolving items that are in the backlog or feature list. It assesses and analyzes the risk, effort, time, and complexity of those requirements to help get an understanding of what it will take to implement them and uses team capability as a way of measuring how long or how many iterations it will take to achieve this work.
Another change in thinking we have to make in agile estimating is a need to focus on outcomes as opposed to the specific activities that will be performed. Another way to put it is that in agile estimating, we are focusing on what we are going to deliver, as opposed to the effort that would be required to produce a particular deliverable.
This is where the concept of story points (which we are going to talk about next) enters the picture and helps us bring some analogous tools and sizing into the process. Eventually, we will factor in hours of effort, but this is not until we drill down into the task level and we are beginning to look at things on a more discrete level for a specific sprint. This doesn’t happen at the start of the project, and the fact that estimation in hours is not done upfront frankly drives some practitioners and students crazy.
Sizing and Story Points
In order to estimate in agile, we need sizing and scoring systems. This speaks to a couple of different things. First, we need to define what we’re going to estimate. Then, we need to develop some sort of scale to express our estimations. When we think about the basis for our estimating, we must also assume that we have a common vehicle to contain the work item or feature. Many agile frameworks or practices settle on the user story as the common method or language for capturing this. We’ll look at user stories in more depth.
When we decide to use user stories, we introduce story points and other “unitless” scales as a way of estimating them. Some agile teams like to use “ideal days” – in other words, the number of days (assuming that the individual or team had nothing else to work on) it would take them to accomplish a particular piece of work. Injecting humor, but keeping in the theme of comparative sizing, there are teams that will use dog breeds as a way of sizing. They will say that a small story is a Teacup Poodle and a very large story is a Great Dane. Some teams may use measures such as a gummy bear or jellybean as in “that’s a five-jellybean story.” Again, the key here is that the team or the organization agrees to a unitless scale that the team can understand and everybody can use.
Story points are one of the most common methods of agile estimating. If we think of something being one story point, that’s a very basic unit. When the story point is completed, you should have one relative unit of outcome. The complexity gets introduced as the team figures out the detail and scale. In other words, the team determines from the delivery perspective what is equal to a single story point, and they begin to establish the scale.
The team needs to think about the relative complexity and what it takes to deliver these increments. They may make up a scale of 1 to 10 or they may come up with a scale of 1 to 100. There is a theory of thinking that the orders of magnitude and the concept of relative sizing take us away from use of large scale. That tends to speak against the scale of 1 to 100 and points toward the 1 to 10 scale. You may be familiar with the Fibonacci sequence – 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, …. The next number is found by adding up the two numbers before it. This is a common way of developing a story point scale. Many practitioners depart the Fibonacci sequence after 21, instead jumping to 25, 30, 50, and so on. At this point, the stories are too large to comfortably estimate using a relative sizing approach and should be considered placeholder estimates for these large stories, which are often called “epics.”
In practice, teams used to plan-driven projects and estimating using a WBS – tasks – hours approach may struggle with or resist this approach. I worked with an agile coach (Andy) who shared with me a method he used to effectively trick some old-school software engineers who swore their work was too “special” to estimate using relative sizing into realizing this approach can be used on pretty much anything. Andy walked the developers through an exercise where they relative-sized the risk, complexity, effort, and duration of preparing various meals: a bowl of cereal, a sandwich, a dozen cookies, macaroni and cheese, a holiday meal.
I adopted and adapted this exercise to use in my agile project management classes and combined it with planning poker (covered next). Andy’s developers (and my students) quickly realized that it was possible to size these endeavors relative to one another. It also served to show how additional refining questions (what kind of cookies? Chunky or creamy peanut butter? Does the definition of “done” include cleanup?) can help teams perform this relative sizing.
Estimating with Planning Poker
An exercise that can be used for relative sizing and estimating in agile project management is planning poker. This is a team process, and as we noted earlier, agile estimating is assumed as a best practice to be a team process. In planning poker, the team gathers, and everybody gets cards (that you hopefully have prepared in advance) that have the story points, the unit of measure assigned to them with the scales. You have a card for 1, 2, 3, 5, 7, and so on, or XS, S, M, L, XL, XXL, or whatever the scale is.
During backlog estimation or sprint planning, the team considers the story that is being evaluated, and when the signal is given, the team turns up their cards. What you hope to see is that the majority of the team has selected the same value, but there will be outliers – people who perhaps give a lower value or higher value.
The outliers explain their thinking: “Sean, why did you estimate that is being a 3 when most of us estimated as 5?” That’s not to put Sean on the spot. Rather, it is to give Sean an opportunity to explain. He may have some familiarity with the work that the others don't have. The same may hold true for somebody who estimates higher than 5. That person is given an opportunity to explain their thinking. They may have unique knowledge that others on the team don’t have. Having shared that information, the team will reset and then flip the cards again. You do this for each user story until you reach a consensus.
The reason we do this as a team is tied to the concept of cross-functional teams in agile. I give an example later of a handyman named Joe and his cross-functional team working on home-improvement projects. Because the person most expert in a particular thing may not always be the person working on it, it is useful to have the entire team’s input on a particular item. For example, one person on the team may be an expert painter, but everyone else also knows how to paint to some degree. Therefore, having the entire team estimate how long it will take to paint a room factors in that the nonexpert painters may be the ones who, after finishing their other work, collaborate to paint one room while the expert painter completes a different room.
Iteration or Sprint Planning
If the team is using an agile framework that includes defined time boxes, typically called iterations or sprints, planning these sprints is a critical activity. Teams bring in a group of prioritized work that they can comfortably commit to and hold themselves accountable to deliver within that timeframe. The ideal outcome of each iteration of work is in increment of completed product that, standing on its own merits, provides value to the team and the organization.
Potentially Releasable Increments
Many agile frameworks talk about each sprint resulting in a “potentially releasable” increment of product. It is important to understand what that means and how that may manifest in different ways in different scenarios. A lot of people new to agile project management think they literally have to have a releasable piece of the product that could be customer-facing for every iteration – that’s not the case. There are different ways of approaching this – a number of different definitions.
To provide examples: A skeletal design is a releasable increment of the product. A single complete feature – if we are thinking about designing a website, it could be that the completed logon page or the completed account setup – could be a releasable increment of the product. A key piece of the infrastructure can be a releasable increment. In the case of a board game that I designed and worked on with a team in scrum master training, the completed game board was a releasable increment of one of our iterations. We could not sell the game board by itself, but it was an important releasable part of the product.
If you’re developing a technology solution, for example, developing the middle tier of infrastructure to support a website that needs to communicate with an enterprise resource planning system – in one iteration, we released the ability for a control on a web page to make a call from the control through the middle tier into the ERP database and return a piece of information. We considered that a releasable increment from that iteration. It may be something like the design and setup of a piece of foundation that’s needed to proceed – this could be a releasable increment or vertical slice.
From the perspective of a magazine or book, you may complete the table of contents and say, “here’s everything that’s going to be in this issue of the magazine,” or all of the chapters that are going to be in our book. It may be the selection of an issue’s monthly theme and identification of everything needed to support it. It may be the story outlines for a particular chapter or for a particular issue. These are all things that would be considered thin vertical slices, releasable increments of the product, things that could be completed as results of an iteration, things to which you can assign a robust definition of done, and test and verify that they are indeed complete per the direction or the definition set by the product owner or the team.
None of these examples could stand on their own as a fully usable product. However, from the perspective of being complete in their discrete entirety and functioning or delivering value on their own, they are complete and potentially releasable (if not totally usable) on their own.
Be conscious of “fast waterfall.” If a team considers the output of early sprints as a progressively elaborated plan prior to starting execution sprints, the team is really doing a “fast” or time-boxed version of plan-driven project management. This is not necessarily a bad thing in the context of agile hybrids, but something the team and the organization should be eyes-wide-open to.
Be aware of when a team is doing “fast waterfall.” If a team considers the output of successive sprints as a progressively elaborated plan prior to execution sprints, the team is really doing a “fast” or time-boxed version of plan-driven project management versus using an agile framework and practices. This is a perfectly acceptable hybrid approach to planning and execution – provided the team is eyes-wide-open to this and understands what they are doing. Given where the organization sits on the continuum, this may be fine in the context of the organization, the team, and their project - it is important that everyone understands what they are doing.
Sprint Planning Steps
Entering a sprint planning meeting assumes that we have everyone from the team inclusive of the product owner present, and that we have prioritized items taken from a fully groomed backlog. It is assumed that we start the meeting with that subset of items that we are considering for completion within the sprint we are about to launch.
The team reviews its capability – if it is performing together for the first time and it’s an early iteration, they are essentially guessing at their capability and capacity – their velocity. If the team has performed together for a period of time, they’ll have an understanding of the amount of work they can commit to within each iteration, and they can use this to estimate what they will be able to do. Based on this, the team reviews and commits to particular features or user stories that they will deliver during a particular iteration.
At this point, the team may do another round of relative sizing using planning poker or some other method that the team, as a self-managing team, has agreed to use. The team revisits only those stories or work items they are considering for the sprint. At this point, the team will break these stories or work items down into their component tasks. I often make a comparison to the work breakdown structure (WBS) in plan-driven project management. In agile, we end up with that work breakdown structure as a planning output, except that we are creating it one sprint at a time, assuming we are using a time-boxed approach in our agile framework and practices.
At this point, it is appropriate to estimate those tasks in hours of effort. This, along with the estimate of points compared with the team’s average velocity and alongside of the available hours within the sprint timeframe acts as a sanity check to ensure that the team actually has the available time and capacity to commit to and complete the proposed work within that sprint.
Another critical process within iteration planning is writing tests or documenting done conditions. Whether we’re talking about software, whether we’re talking about product development, or whether we’re talking about the development of a new magazine or website, on any project, there needs to be definition of, and agreement on, what represents done. This is critical, because at the end of each iteration, the goal of most agile approaches is delivery of completed components – things that can be considered done, and that on their own deliver some kind of value to the project and the organization.
Once the team has identified its work, completed its estimating, and agreed on definitions of done, it is time for the team to make a final commitment to completing this sprint backlog of work. At this point, it is important for the team to have some clear way of assuring everyone’s commitment and agreement that the work is correctly defined and achievable.
From my friend and agile coach Andy, I learned the Fist of Five technique. In my current job as CIO at a community college, we have a technique we call the checkout. Both techniques involve showing one through five fingers or a thumbs up, sideways or down as a way of indicating full, partial, or no commitment to the plan for the sprint or solution to the problem. If a team is not fully committed, it is critical to resolve those elements prior to launching the sprint.
Demos
Another process common to most agile methodologies is a demo at the end of each iteration. Each iteration is to yield a completed piece of work – something that on its own provides some kind of value to the product owner or organization.
This is an opportunity for the product owner or other stakeholders within the organization to ask questions and to provide feedback on the deliverables. There is some indication from the PO of formal acceptance of the completed work or portions of the completed work at the end of the demo. This is also fed into the retrospective, which we will talk about shortly.
The best demo story ever: Ecolab of St. Paul, MN, was an early adopter of CloudCraze as its ecommerce system. They had envisioned a phased implementation, and the first phase was to simply stand up the core product and start taking orders. This was to be an eight-week release with two-week sprints. After some initial planning work and team formation, we put our small team in with Ecolab’s team for the first two-week sprint.
As this two-week sprint came to a close, I worked with the client lead to orchestrate the first demo – it was going to be kind of a big deal, and everyone (including me) had a part in the demo. We were advised that some senior Ecolab executives would be joining via web conference, and the client lead noted we should pay particular attention to any questions or comments from “Sam,” a senior VP.
The demo went very well – every feature worked, everyone’s part in the demo went well, and there were general good feelings all around. Sam, the senior VP, thanked everyone and then asked for the slides showing the product features. There was total silence for a moment – then, I realized that Sam thought he had been seeing “slideware.”
I spoke up and said, “these aren’t slides – you’ve been seeing working software and features delivered after a two-week sprint.” After a brief pause, Sam said, “can we go live on Monday?” – followed by laughter, smiles, high fives, and so on. Clearly, the demo had gone very well, and this served as further motivation for the project team. In fact, it had gone so well that Ecolab approved adding another business unit to the ecommerce project before the original project was complete.
Retrospectives
Ongoing learning is a key process. Agile frameworks include the team’s evaluation of its performance at the end of each iteration. This is critical so that the team understands what it’s doing well and things that are not working so well. With this information, they can make necessary changes right away with their next iteration to ensure they consistently improve their performance – remember – plan-do-check-act.
Having an understanding of their performance informs ongoing and iterative planning. To put it another way, the project team has a better perspective or better picture as they complete each iteration. That in turn gives them an understanding of what the remainder of the project might look like so they can update the plan. They can provide an overall update to the product owner, stakeholders, or sponsors of exactly what the remainder of the project might look like. Again, there is the understanding that the overall release plan changes and needs to be fine-tuned with the completion of each iteration.
Release Planning
The release plan helps the project team and the performing organization understand what the overall release structure looks like – what they can count on. The release plan gives us an understanding of when we will be able to release something of value – whether it is to the marketplace, something available for purchase to generate revenue for the company, or something of value that is released internally to the company that gives value by providing the capability to the organization, or perhaps efficiencies that did not exist before.
A time-driven release plan shows how many iterations are possible within the prescribed timeframe based on the amount of estimated work to be done. A feature-driven release plan provides an initial idea of how many sprints will be necessary to complete the work determined necessary to deliver a minimum viable product.
At the core of release planning is the constant prioritization of features, functionality, and user stories so that the deliverables that will provide the highest value are at the top of the product backlog and reflected as such in the release plan. It is also critical that the project team and the product owner are working together and discussing mandatory dependencies as well as work items that are complementary. Overall release planning ensures that the product being built in this incremental way is coherent and that the order in which work is being done makes sense as well as ensuring highest business value items are completed as soon as possible.
Summary
In this chapter, we have discussed elements of agile and agile values that are important to know in order to help you understand how agile should function. Adaptive, flexible, iterative, and fast are all the elements of agile that are part of its values.
We discussed agile life cycles. We discussed how one version of the agile life cycle begins with what is essentially a period of waterfall initiating and planning work prior to starting sprints – a hybrid, AgileFall-type approach. We also looked at the basic Scrum life cycle and how it starts with a vision and backlog, followed by sprint planning and the sprint, and ends each sprint with the sprint review including a demo and retrospective.
In addition to providing foundational information about agile and scrum, we noted examples where practical hybrid application of these techniques would be the realistic approach.
Chapter 6 will discuss how agile teams should be formed and used and examine communication practices that often suggest these teams sit together. We’ll look at whether that is always realistic and how to address this given the myriad of communication tools available to global virtual teams.