Planning Fundamentals

It is important to understand that the duration of the project is not equal to the duration of its constituents. In Lean manufacturing, there are three distinct terms.

In terms of an assembly line, the Takt time describes the required product assembly duration end to end. Takt time is defined as the average time interval between sequential production of two similar units, for example, two engines on an automotive assembly line, if those are produced sequentially. It is also referred to as a delivery heartbeat that we’d like to synchronize with the product consumption cycle.

Cycle time is the time it takes to complete a cycle of action, from start to finish. An example is an assembly of a car engine, start to finish.

Lead time is the total time between the initiation and completion of a project or a task.

Usually, lead time includes all sorts of overhead, referred to as “waste” in Lean manufacturing. This includes waiting between the dependencies (e.g., testers are available but there is no code available to test yet or code is available but there is no test environment to deploy it to); resources, especially those with rare skills (the sole UX designer on a team got sick or took a vacation); requirements not available; design decision that has not been approved; and so forth. According to different sources, the actual time when team members work on the project tasks is up to 60–85% of the lead time for a medium-size project, while up to 40% is the waste due to mismanaged dependencies, resource constraints, blocked process flow due to quality issues, unbalanced supply, delayed decision-making, and so forth.

Incremental Planning

What does this all mean from the project planning perspective? Most importantly, it means that it is dangerous to predict project duration based on the duration of its tasks. In traditional project management, this is sometimes done by adding a specific extra time and budget to the project plan, up to 50% of project duration for the most complex projects with multiple dependencies. This differs by industry. For construction projects, for example, project managers usually add 5–10% of the total budget and duration to determine contingency. However, this number is arbitrary and is never accurate since it is not clear when this contingency should be added to the project: in the beginning, in the middle, or prior to completion. At the same time, it is known that there will be “known unknowns,” “unknown knowns,” and possibly even “unknown unknowns” that will influence the project through its life cycle. How is this issue addressed in Agile?

First, the answer is incremental planning. The plans are refined given the “cone of uncertainty.” The closer to the project completion, the more predictable the timelines are. Given that the cone of uncertainty in Agile is initiated with every iteration, there is full predictability related to each iteration being achieved at the end of this iteration. This allows for increment-managed predictability and increment-based planning.

Second, the answer is replanning, which happens in every iteration. Based on the outcome of each subsequent iteration, the plan is revisited, and the empirical data observed in prior iterations is used for future planning. For continuous delivery frameworks, such as Kanban, this planning adjustment is happening in a continuous manner so that all the unknown unknowns can immediately be taken into consideration. The sense of urgency in these implementations can be compared to the war strategy that allows an army to win the war.

 Topic for a Group Discussion

In groups, compare static planning done in traditional project management and iterative planning done in an Agile environment. What are the benefits and drawbacks of each approach? Why do agilists say that traditional project management promises stakeholders peace of mind but fails miserably most of the time?

Multilevel Planning

From this perspective, Agile planning includes multiple levels, moving from long-term planning (company’s mission and vision) to short-term time planning (daily meeting with Scrum where participants discuss what they were doing yesterday and their plans and impediments for today). These levels are also referred to as an “Agile planning onion” because they can be represented as a set of layers from long term to short term, resembling the process of peeling a metaphorical onion (see Figure 7-4).

How to interpret the planning onion? In Agile, planning happens continuously at multiple levels. It starts with the company’s strategy, aligned with its mission and vision. Many companies use OKRs (see Chapter 1) to establish long-term business goals focused on customer success, such as customer satisfaction increase or business targets. These goals are established for longer-term intervals, usually for one or several years, and reflected in product roadmaps. They focus on “why” rather than “how.” If a company launches a new product and this product does not perform as expected, it can pivot by launching another one or enhancing its existing product portfolio within the same set of OKRs.

The next level is the product portfolio. For a bank, the product portfolio would include all the products and services it provides – checking and savings accounts and related services, loans (home, auto, and any others) investment products and services, and so on. It is an important decision on which products to provide as part of the portfolio, which to sustain, which to sunset, and which new ones to launch to disrupt the current product line. This level of planning is aligned with the strategy and is also usually done long term. It affects product roadmaps and defines execution.

For each of the products within a portfolio of products and services, there is a product roadmap. The product roadmap reflects the product life cycle. Each product experiences multiple phases within its existence, which need to be managed, from its inception, through design, development, adoption, maintenance, and sunset. This cycle varies for each of the products and depends on its adoption rates, business success, customer feedback, overall market status, company policy, and a lot of other factors, which directly or indirectly affect it.

While the concept of a project is not directly relevant in an Agile environment, it is frequently used in large enterprises to define major feature releases. Frequently, a concept of the release is confused with software deployment. It is important to distinguish between the two: software deployment results in making a software system or any relevant functionality available for use. The general deployment process includes multiple activities that result in the ability of the customer to use the software. Those can be frequent, especially when those are automated and deploy minor changes on a continuous basis. For example, Netflix engineers deploy code thousands of times per day. Google deploys sofware many times per day, and so do many other companies that implement the DevOps strategy of continuous deployment. However, a deployment that makes specific functionality possible does not constitute a major feature release.

A major software release means a new version of the software that includes significant architecture changes or new features on top of the original functionality. For example, if an online bank provided checking accounts only, providing savings accounts would be a major feature. Even in an Agile environment, those are usually referred to as projects because there is a stated business goal and defined (though flexible) scope, and frequently, there is a timeline, whether it is based on vendor agreement, business expectations, or a new regulation. These releases are usually announced to customers and stakeholders in advance with a specific commitment. This can be compared with major releases of new iPhone models and related iOS, starting from the release of the original Apple iPhone in 2007. These are usually listed as a milestone on the product roadmap.

In order to deliver the functionality of the product within the forecasted milestones, many Agile frameworks use incremental delivery where planning is done in short time frames, referred to as increments (or Sprints in Scrum). As we discussed, Sprint duration in Scrum is usually two to four weeks. During this time, delivery teams make a commitment based on their prior capacity and their delivery history. Using this empirical data, they are able to accurately predict the work they will be able to accomplish during the Sprint. Based on the results of the Sprint, they adjusted their estimation for the upcoming one. Because of the consistency of team composition and the availability of empirical data for analysis, the accuracy of Sprint planning on mature Agile teams usually exceeds 80%. We will discuss this process in more detail further in this chapter.

However, as much as the team is able to predict its delivery within an iteration, known unknowns and even unknown unknowns may happen. These include new market entrants that disrupt existing competition, government regulations, changes in vendors, staff attrition, and many other unexpected circumstances. For this reason, team members meet on a daily basis for a brief daily meeting (referred to as “Daily Scrum” in Scrum) to align on all the tasks they are working on and support each other in resolving impediments. This represents the daily level of planning, which allows for day-to-day alignment between the team members. In larger organizations, Scrum Masters for every team meet for Scrum of Scrums to resolve intrateam dependencies and remove shared blockers.

 Topic for a Group Discussion

It is a known belief that there is no planning in Agile. In every iteration, a new plan is created, which can change at any time. This makes planning look fluid and unreliable. The timelines seem to shift all the time based on the current state, while in a traditional project management plan, every single task is accounted for, durations are defined, sequencing is provided, and the overall timelines are clearly stated, along with the people who are responsible for working on them. Discuss this statement in your group: Do you agree with this point of view or disagree with it, and why?

 Tip

Planning does not happen in a vacuum. It is important to establish a credible intake process – how the ideas are captured, assessed, prioritized, and sequenced. Planning will not be successful if the intake process is not well thought through, communicated, established, and followed through. It usually depends on the organizational culture, size, industry, nature of the business, and other parameters. We covered different intake and prioritization techniques in the first part of this book on how to build the RIGHT product. In any organization, thoughtful and relevant intake is a prerequisite for any planning to be successful. Ensure that the intake and prioritization process is well defined before engaging in planning activities.

To summarize, there are multiple simultaneous levels of planning in Agile, also referred to as “planning horizons”:

 Five key questions to review:

Estimation in Agile

In order to achieve predictable planning outcomes, Agile teams estimate their work based on their capacity. In Agile, team capacity is referred to as “velocity.” The definition of velocity is the number of units of work completed in a given time. The units of work are usually measured in abstract measurement units, referred to as “story points.” A story point in Agile is a metric used to estimate the effort required for implementing a given user story. This includes the work required to complete the story and any approvals and decisions that have to be completed as part of the implementation, any external dependencies, and other relevant factors, such as the lack of team members’ experience with this specific type of work, or limited subject matter expertise in the area.

Story points do not equate to hours directly since the effort does not include the time required to produce the output. It accounts for all the elements discussed before, including dependencies that need to be satisfied, the learning and experimentation that has to happen, solutions involved, and so on. Given that story points provide a holistic estimation of the effort, they are much more accurate than time estimates that take only “known knowns” into account by adding the time required to complete each of the tasks that have been identified.

Agile estimation is done by the whole development team (developers, designers, testers, etc.) so that each team member brings their own perspective of the product and the work required to deliver each user story. For example, if a Product Owner would like to make a simple screen change to one of the user profile fields, it may require performing full regression testing for the whole product, given that this parameter is being used throughout the whole application, and the testers would be able to indicate that.

A helpful technique in facilitating story point estimation in Fibonacci numbers (1, 2, 3, 5, 8, 13…) is referred to as the planning poker. All team members select a playing card with the story point estimation they suggest for each user story. If all team members vote for the same value, it is assigned to the user story. If not, then team members with the highest and the lowest score provide their reasoning, and the whole group votes again until they reach a consensus. If they do not reach a consensus, different teams have their own agreements on how to resolve minor discrepancies. For example, some teams always agree to honor the higher estimate if there is only one degree of difference after several conversations and voting cycles (e.g., 2 and 3 or 3 and 5). During the conversation about user point discrepancies, a lot of helpful facts get discovered that some team members are aware of and some are not. It is a helpful collaboration technique and not just an estimation technique.

Since the length of each iteration (Sprint) is fixed, each team develops its average velocity over the course of several Sprints (usually, 2–4 latest Sprint average is most meaningful). For example, if team A had a velocity of 18, 20, and 22 story points over the last three Sprints, respectively, without changes to the team composition, they would commit to a maximum of 20 points for their next Sprint. However, if one of the days of their two-week (ten working days) Sprints is a holiday, they would commit to 18 story points only for that Sprint. The same type of logic applies to vacations, sick days, and other changes in team capacity or composition.

 Tip

A good illustration of relative estimation is a fruit salad exercise. In this exercise, team members are presented with identical set of fruit (apple, pear, grape, strawberry, etc.) and a “Definition of Done” for a fruit salad; for example, all fruits are washed, peeled, deseeded, and cut into salad. They are informed that their Sprint velocity equals 20 story points and asked to estimate the fruit they will be able to make a salad from. Usually, they start with a grape or a strawberry estimating those at 1, then apple or pear at 5, and, finally, pineapple at 13. This leaves them with a combination of one grape, one strawberry, one apple, and a pineapple, or another variation that is estimated up to 20 story points.

The most important rule is to learn from past estimates and adjust the current estimates based on this learning.

In this case, when a new user story needs to be worked on, it is estimated by comparing it with the ones on this banner. For example, if there is a need to search by brand, the effort may be seen as similar to keyword search (hence estimated as three story points). If there is a need to delete the profile, it may be seen as similar in size to updating it (hence estimated as two story points) and so forth. The nature of the work performed may be completely different, but the effort related to getting it done is seen as a similar one.

The next question: What happens if the set of the stories pulled into a Sprint one by one does not fit within the velocity estimated for the upcoming Sprint? Let’s review a sample product backlog structure with estimates below:

If the team’s velocity for the upcoming Sprint is 18, the team pulls stories from the top of their backlog into the Sprint until they reach 18. In this case, user stories 1–5 get pulled and reach the total estimate of 15 (5+2+2+3+3). The next user story is #6, which is estimated at 5 story points. In this case, team B will skip user story 6 and pull user stories 7 and 8 into the Sprint. If the Product Owner thinks that user story 6 is the absolute priority over 7 and 8, they can “decompose” user story 6 into several smaller size stories. For example, if user story 6 is to search for a product, they would decompose it into search parameters (by name, by category, by manufacturer, etc.). For user forms, they can decompose by “create, read, update, and delete” (CRUD) functionality and so forth. In either case, the story has to satisfy INVEST criteria.

Exercise

Split in teams and try an equivalent of the fruit salad exercise with dogs: Chihuahua, Poodle, Beagle, Bulldog, Collie, Labrador, Golden Retriever, Great Dane, Mastiff, and Greyhound. The Definition of Done in this case is adopting the dog, so the comparison will be between sizes, temperaments, cleanliness, and other relevant features.

Tip

There are five steps in Sprint planning:

The approach to estimate feature size in story points and convert them to time duration at the time of estimation has proven to be inaccurate because it requires time-based estimations, which are usually inaccurate because they do not account for all the effort required for delivery. At this level, aggregated story points are the best measure of predictability.

During release planning, an Agile team schedules delivery into iterations, or Sprints. They use the prioritized user story backlog to move user stories (and sometimes epics, especially for the further timelines) into a specific iteration, based on the effort required and team velocity. This is described in Figure 7-5.

A good example of medium-scale planning in Agile is the PI (program increment) planning in the Scaled Agile Framework (SAFe) (4). It is a cadence-based planning event that aligns all Agile teams to a shared mission and vision. It implements the Agile Manifesto principle that “the most efficient and effective method of conveying information to and within a development team is a face-to-face conversation.” PI is a large-scale planning event based on the premise that people who do the work plan the work. PI planning has a standard agenda defined in SAFe. It includes a presentation of business context and vision, followed by team planning breakouts. In these breakouts, the teams create their iteration plans and objectives for the upcoming program increment (many companies plan in quarterly program increments).

PI planning is an asynchronous, usually two-day, event with the participation of all members of delivery teams within the program, though it is often extended to accommodate remote planning across multiple time zones. It allows establishing face-to-face communication (frequently virtual) across all team members and stakeholders. In addition, it aligns development to business goals with the visual context, vision, and program objectives; identifies dependencies across multiple teams; provides an opportunity to discuss high-level solutions and make decisions on architecture, UX, and other guidelines and solutions that match demand to capacity to minimize work in process (WIP); and allows for fast decision-making.

Its outputs include committed PI objectives (a set of SMART – Specific, Measurable, Achievable, Realistic, and Timely – objectives) that are created by each team with the business value assigned by the business and the program board highlighting the new features and forecasting delivery dates, featuring dependencies among teams, and identifying relevant milestones.

 Tip

To ensure planning accuracy, it is important for each team to establish their Definition of Ready and Definition of Done. The Definition of Ready provides a set of criteria that are sufficient for a user story or a task to be taken into a Sprint by a Scrum team. For example, if acceptance criteria for a user story are vague, it may result in a lot of confusion during the Sprint, and the team won’t be able to complete it as expected. Similarly, the Definition of Done provides a set of clear expectations for a user story about what does it mean that it is considered done. Frequently it includes both process-related items (all acceptance criteria are satisfied, the user story is accepted by the Product Owner, and similar expectations) and technical items (the user story is deployed into the production environment and the functionality is integrated with downstream systems or nonfunctional requirement related to performance and resiliency).

Large-scale planning. There is a belief that Agile is failing in large organizations because strategic planning conflicts with short planning horizons in Agile. As stated in the Forbes article “Is Agile Failing Long-Term Planning” [4], Pradeep Ittycheria, a Forbes Technology Council member, argues that “there’s no doubt about it: Agile works. There are many stories of companies that have benefitted from going Agile. However, Agile is failing to meet expectations in larger organizations; in many cases, it is failing because established strategic planning conflicts with the notion of a Lean enterprise and Agile planning (with very short time horizons).

In my experience, capital allocation decisions are typically made six to twelve months in advance. Investors and management teams want to carry out strategic planning for an entire year. Financial targets are yearly targets. Planning for a year in the future is not a time horizon that fits the scheme of Agile.”

This opinion is a well-known Agile myth. As we saw from the “Agile onion,” there are multiple concurrent levels of planning in Agile, which are executed in parallel. Specifically, for the long-term estimation and planning, a product roadmap is usually created based on prior data and subject matter expertise, which is broadly shared with the stakeholder for their input and feedback. No matter how Agile a project is, there is always a need to manage scope, resources, and time when planning a strategic vision. There are four steps to achieve an accurate long-term estimation in Agile:

Step 1: Start with the big picture. Based on OKRs, define actions and milestones. Setting your high-priority themes will help you focus your time and energy on the high-priority work that matters. For large-scope estimation in Agile, T-shirt sizes (Small, Medium, Large, XL) are used more frequently than Fibonacci numbers. These sizes usually translate directly into the timelines.

Step 2: Identify high-priority work items. Identify the most important items to be completed and achieve consensus with the stakeholders on this prioritization.

Step 3: Decompose these items. In this step, we need to break the work for the larger initiative into more consumable work items, such as epics. This will provide a granular view into all the work that needs to be completed and makes estimation tangible.

Step 4: Create a roadmap. Establish a roadmap based on the decomposed items by comparing them with previously delivered work units. Because estimating requires knowledge of similar efforts in the past, it is helpful to compare this project to the ones previously completed. The product roadmap outlines the progress of a product over time. Iterations are translated into timelines, and timelines indicate feature delivery to the customer. A product roadmap template is presented in Figure 7-6.

Once the roadmap is agreed upon, it is shared with the entire product team so that everyone understands the vision, deliverables, and dependencies. It becomes the delivery guideline, which is being updated on a regular basis to reflect progress with delivery and changing business priorities. Product Owners usually own the roadmap, whether it is a static representation of feature delivery, as shown before, or a dynamic roadmap created with one of the specialized tools, such as Jira, ProductPlan, Aha!, Productboard, Trello, and Roadmunk.

In sum, Agile enables companies to execute planning not at the project level, as in the traditional project management – it allows to plan at multiple levels in parallel, continuously narrowing the “cone of uncertainty” to achieve predictability and allow for live updates. In addition, the planning is not happening project by project using arbitrary start and end dates defined by project managers; rather, it enables planning at the value stream level. A value stream is a term coming from Lean that shows how customer value is being created – from the start of the value chain until the delivery to the customer. It allows to establish a customer-centric delivery process and plan for the delivery of products and services rather than planning for a specific set of deliverables that may or may not achieve customer and business value.

Five key questions to review:

1. 1.

   What is a product roadmap?

    
2. 2.

   Why multiple planning horizons are possible in Agile?

    
3. 3.

   Who is responsible for creating and maintaining the roadmap?

    
4. 4.

   What is the difference between Sprint planning and PI planning?

    
5. 5.

   How Scrum teams plan their dependencies on each other and external providers?

    

 Tip

Longer-term planning requires that there are business priorities defined for a noticeable period of time (at least one quarter) and there is enough consensus on the features to be delivered and the priority of their features. Sometimes, especially in the case of startups or the companies launching new product lines, this level of certainty is not possible. This does not mean that these companies should not strive to provide longer-term visibility. A good practice in this case is to establish “rolling wave” planning, which means that features and feature sets are planned as they arise, with all the prerequisites and dependencies being properly planned and any risks being assessed and mitigated.

Key Points