6 Agile planning

“Plan for the difficult while it is easy.

Act upon the great from the beginning of the minute.

All difficult affairs must be taken care of when they are easy.

All great accomplishments must be performed from the small tasks. Lao Tzu”

Planning is an area that often causes problems and misunderstandings when discussed in conjunction with Agile. The Agile Manifesto statement ‘Responding to change over following a plan’ can be unhelpful when interpreted to mean that planning is not necessary. I have even heard of organizations using this statement as a reason for not planning at all in Agile projects. Conversely, it is also used as a reason for organizations not adopting Agile, believing that Agile initiatives are unplanned and therefore likely to go out of control.

In fact, planning is central and key to Agile. The old saying ‘Failing to plan is planning to fail’ is equally true for Agile initiatives. Planning allows us to set expectations, understand our commitment in terms of time, money and resources, and determine whether we are succeeding.

The Agile Manifesto statement above can be interpreted in two ways:

• Firstly, we accept that the further into the future we try to plan, the less certainty there will be. This means that detailed planning is only worth doing for the immediate future. This is often called ‘just in time’ planning in Agile. Experience and new information gained from carrying out the initiative will clarify our plans into the future.
• Secondly, we also accept that change is inevitable and plans will need modification based on these changes. So we may not follow the plan in its original form, but we understand the reasons and impact of changing it.

So planning is still necessary in Agile approaches. Initially, we plan at a high level for the whole length of the initiative. Otherwise, how can we know what to deliver incrementally? How can we judge the costs and potential delivery times? Then we plan in more detail to cover the next, relatively short, period.

Planning is not an activity undertaken only by managers. As a rule, plans should be created collaboratively by those on whom the plans have the most impact, and who have the relevant knowledge and skills to be able to plan. For example, Agile teams will plan for themselves the best way to deliver desired outcomes within a given period; customers will plan the best times for implementation into their organization.

All stakeholders should be involved in planning, particularly those who need to contribute resource or who will use the resultant capability.

The following sections describe how to carry out Agile planning. Although you may leave the fine detail to the experts in your organization, you should definitely be involved in the process. You may find it useful to understand exactly what is going on and the terms being used. Equally, do encourage the planning experts in your organization to read these sections.

6.1 Planning approaches and levels

We will discuss two approaches to planning – one using portfolios, programmes and projects, the other using a continual delivery paradigm.

6.1.1 Portfolios, projects, programmes and planning horizons

Agile planning will occur at different levels, as shown in Figure 6.1. At the lower levels, the planning timescales are shorter and the plans are more detailed. At each level of planning, it is important that the right people are involved. At the portfolio level, for example, senior stakeholders will need to be involved as they will need to consider the impact and benefits of various initiatives and how they may resource and implement them. At the project level, the teams that will do the work should do the planning. Throughout the process, it is important that all relevant stakeholders are involved – from customer to supplier, from support to operations, from business owner to developer.

6.1.1.1 Strategic and epic level

Initiatives at this level are often called ‘epics’ in Agile. Strategic or epic portfolios may cover many years. At this level, the detail is normally no more than a description of the initiative, any statutory or business-critical implementation date and possibly an estimate of the duration. This is virtually the same as with traditional portfolio planning. When we are using an Agile approach, we want to ensure that the right initiatives are undertaken at the right time so that the organization can benefit as early as possible. Consequently, reviews of the planned portfolio need to happen more often, so that initiatives can be reviewed and started as soon as possible.

Some investigation of the epics may reveal subsets that can be delivered separately. Working on sub-epics will mean that the organization can benefit earlier and does not need to commit as much resource, time and budget in one go. In fact, the sub-epics will become items in the portfolio in their own right. The full epic will not be delivered until all parts of it are completed; however, as the sub-epics are delivered, it may become clear that other parts are no longer required or add limited value to the organization.

6.1.1.2 Programme level

Programme-level planning will concentrate on defining the potential steps required to meet the goals of the programme. In Agile programmes, each step should be as short as possible (realistically about nine months). Steps may also be planned to fit with existing governance procedures such as budgeting cycles. Each step will introduce into the organization new capabilities which lead to benefits, and consist potentially of many projects and activities.

When you work with an Agile mindset, only the first step needs to be approved, and then you can plan this step in more detail – that is, exactly which projects and activities are required and the timescales around them. Using the ‘just in time’ principle, the details of the projects and activities within the first step are not yet planned.

6.1.1.3 Project level

Agile project planning takes a similar approach to that of programmes. The structure and outputs of the project are planned at a high level, enabling the project to be divided into increments, which can deliver value into the organization.

6.1.1.4 Increment level

Although not prescriptive, increments may be up to three months long. The detail of an increment is not planned until it is about to commence, but the high-level features of the increment are understood. At the appropriate time, an increment will be planned into timeboxes or sprints, which normally last no more than four weeks. The idea is that each of these low-level timeboxes produces something that could be used. Some organizations go as far as implementing the outputs of these low-level timeboxes, but this depends on whether the organization can cope with this frequency of delivery.

6.1.1.5 Timebox and sprint level

The lowest level of planning is arguably the timebox or sprint. ‘Timebox’ is a slightly misleading term, as it can be construed as any fixed length of time in which something is done. So a project or a programme itself is executed within a timebox. In this section, we use the term ‘timebox’ to mean the fixed length of time in which a low-level feature is realized.

Timeboxes typically last up to four weeks. At this level, the team is responsible for the realization of specific features and will plan the detail of the timebox. The plans are likely to be very visual and the team has total control over them, within the constraint that the timebox is to deliver a specific set of features.

6.1.2 Planning using epics and features

In some Agile approaches, an organization will have a backlog of epics. These are prioritized as appropriate and the organization simply works through them, breaking epics down into features that can be delivered. This is a continual process which removes the need for temporary structures such as programmes and projects. Teams remain assigned to product areas and work on epics in those areas. In many ways, however, the planning approach remains the same. For planning purposes, we could equate initiatives as follows:

• Epics to portfolio or programme level
• Major features to project level
• Features to timebox or sprint level.

The approach to delivery is different, however; the work is not structured in projects and programmes. Features are created using a concept similar to that of a production line. At any time, teams have the responsibility to deliver a set of features within a defined timebox. As soon as these are delivered, they plan and start the next set. Lean principles can be applied to ensure the process is efficient, and to help coordinate multiple teams and keep the focus.

This approach can seem alien to those used to working in a project world. It also assumes that an organization’s portfolio of initiatives is always centred on a defined product set, and that teams supporting those product sets will remain fairly static. I have already discussed some of the potential pitfalls of this approach in this publication. At the same time, working in this way can lead to a clear sense of vision and purpose. The organization sees continual improvement in its products and services, and the teams get a sense of achievement.

The approach may not work as well when the organization wants to introduce new capabilities that do not involve existing products or services. Ironically, it may actually stifle innovation!

Some of the Agile methods discussed in Chapter 4 are well suited to this way of working, particularly LeSS and SAFe.

6.1.3 Prioritization

In order to add value early and often, we need to prioritize our initiatives at all levels. We need to understand what will give most value and can also be delivered early, as some benefits may not be realizable straight away. Understanding the value will also enable us to determine which items add little or no value, or those where the costs of realizing them outweigh their value. We can then decide that such items go to the back of the queue and may never happen. This prioritization method is used in conjunction with the planning processes where the emphasis in Agile is on fixing time and cost, and being flexible about benefits.

Such prioritization enables us to accommodate change as the value and cost of new and existing items can be compared to understand whether to remove them or incorporate the change.

6.1.4 What do Agile plans look like?

The format of plans can look very different in Agile. In general, we are planning for delivery of features. Task-based planning does not make sense; we want to plan for teams to work on producing something that could be used within the organization. It will be up to the team to decide exactly what to do to achieve it.

Gantt charts have traditionally been used to represent plans. Although there is no reason why they cannot be used with Agile, the information they provide is limited and could be confusing. Figure 6.2 shows a typical Gantt-based Agile plan.

The detailed team plans are often held in an information ‘radiator’; that is, a visualization of the plan, best displayed on the walls of a team room for all to see and potentially update. Kanban boards are an example of a visualization and are explained in Chapter 7. They can be displayed on whiteboards using sticky notes, or electronic tools can be used. There are also tools that can consolidate the information, and even present it in Gantt chart format if required.

6.2 Resource planning

Agile initiatives generally deliver more frequently and over potentially shorter timescales than more traditional approaches. To accommodate this, resource management activities need to be optimized and allocation cycles shortened. This means that resource allocation reviews need to be more frequent – for example, every three months. This timescale is short enough to accommodate most Agile initiatives, but long enough to ensure that teams are stable and that the work can be completed. There are a number of actions in each review:

• Review the previous three-month period and learn from it
• Decide on the items to be resourced in the next three months, prioritized (using a technique such as MoSCoW). It is important to say that this includes future increments of currently running initiatives
• Assign resources appropriately, ensuring they can self-select as much as possible
• Plan in outline, for example, the next nine months, knowing the detail may change.

The resulting plans can be fine-tuned and reviewed even more frequently – for example, weekly or fortnightly. This will allow change to be accommodated where new initiatives have to be done or where planned initiatives are no longer feasible in the time period.

6.2.1 Deciding priorities

Frequent prioritization is necessary. High-level priorities will be defined for the business year, having been identified, for example, during the budgeting cycle. Figure 6.3 shows a potential model for regular reprioritization.

These priorities may change as the business environment changes. Although these high-level items need to be delivered during the year, some may be more important during a specific part of the year – for example, for the next quarter. These priorities can be determined immediately before the quarter starting, using techniques such as MoSCoW. At this level, the importance and urgency are also matched to the business’s ability to provide adequate resources and support.

6.2.2 Resource allocation

An important concept in Agile is that of self-selection. Individuals come together to form the team for a particular initiative. Generally, the team is more motivated, having itself chosen what it wants to work on. Therefore, it is worthwhile to provide resources with a list of upcoming initiatives and give the team the chance to ‘volunteer’. However, there are some points to consider:

• Are all the required skills provided in the self-selected team?
• Is the appropriate knowledge covered?
• What about important initiatives which no-one wants to do?
• How can skills and knowledge be transferred?
• What learning and career opportunities are available?

The final allocation has to balance, providing the best team for the job while ensuring organizational and individual needs are covered.

Agile approaches work best when resources are assigned to an initiative on a full-time basis. This may not always be possible in an organization, so a good compromise is to assign resources to no more than two initiatives at a time, in which case the balance between the two has to be clear.

6.2.3 All resources

When undertaking resource management, you need to consider all resources. This includes personnel from all business and operational areas, as well as non-human resources. For convenience, I will call all these a ‘resource’. Agile projects are fast-moving and it is imperative that environments are available as and when necessary to avoid delays. For example, there is no point in having a team ready to lay track if the tunnel-digging or track-laying equipment is not available.

6.2.4 Part-time resources

Part-time resources can be people or other resources such as pieces of equipment.

For various reasons, normally related to the skills, knowledge or function, some resources are only required at certain points in initiatives, but across a number of initiatives. Although a notional allocation can be made to initiatives at the portfolio planning stage, the actual allocation has to come when the Agile initiatives have been planned in more detail. There needs to be coordination across the initiatives and with the appropriate resource to ensure availability exactly when needed. At this level, as long as the people (or the owner of the resource where it is equipment) understand and are committed to each initiative, they should be empowered to manage their own time with respect to working on them.

6.2.5 Less is more

When planning, it is often better to plan to do less at one time. Taking on too much simultaneously creates complexity and leads to delays, communication problems, and poor outcomes. Given the short cycles within Agile, it is better to complete small increments and then move on sequentially to the next increments.

6.3 Planning for incremental delivery

In order to deliver value early and often, we have to understand how the work can be partitioned to provide components that can be introduced incrementally into the organization. We also have to determine the benefits that will be realized by introducing them.

When considering this, there are three basic scenarios to look at. In the first, we are enhancing or changing something that already exists. The second concentrates on the introduction of something new. In the third we may be replacing an existing capability.

6.3.1 Enhancing or changing an existing capability

Since the capability is already in place, there will be an existing set of infrastructure, systems, organization and processes that support the capability. This is true whatever the capability – from an IT system to a rail network, from an office building to a nuclear power station. In this scenario, we are either changing parts of it or adding new features to it. As such, we may be able to make incremental changes very often as the changes can be applied to the existing organizational infrastructure.

Sometimes changes may rely on amendments to a fundamental part of the organizational infrastructure (for example, database schemas in an IT system, or overhead cables in a rail network). When planning for incremental delivery, we need to ensure that these fundamental areas are created and delivered early, so that small increments can follow.

The effect on the organizational structure, business processes and training requirements will need careful consideration. A series of small changes may cause too much business disruption; consequently it may be better to implement a series of changes at defined release points. This may adjust the order in which the various changes are to be produced. For example, there may be an optimal order of implementing changes so that minimal rework is required in implementing future changes.

All of the above points can be considered during early upfront planning, showing how vital this planning is.

6.3.2 Creating new capabilities

When we consider creating new capabilities, we must accept that there is a minimum set of features required to make the capability feasible. If any of these features is missing, the customer cannot carry out their job at the quality and performance levels required. In Agile this set of features is commonly called the ‘minimum usable subset’ or ‘minimum viable product’. The meaning is the same – without this set of features, the capability is not possible, not safe to use or just causes too much work to be usable. In future, I will refer to this as the ‘MUST’.

It therefore does not make sense to implement subsets of the MUST. However, this does not mean that there is no need for incremental subsets. Such subsets will define self-consistent components of the whole solution that can be developed and ‘signed off’ and then are ready for implementation as soon as the other necessary components have been completed. In some cases, giving users access to these components may add value. For example, there may be advance preparatory work they can do (such as loading data), and they can provide feedback on the use of the component early enough for any major issues to be resolved without affecting the project timescales. Whether the component is actually used or not, it can be viewed as completed, and teams can safely move on to other areas.

The preferred situation is to have components of the solution that contribute to the final overall capability, but also add value in their own right. Although they do not fulfil the total capability, they will be of immediate benefit.

Whichever situation is true for the initiative being planned, you need to give due care and attention to these subsets as they ensure that you use the Agile approach to its fullest.

6.3.3 Replacing capabilities

Replacing capabilities combines aspects of creating new capabilities and enhancing existing or changing existing ones.

Increments may replace parts of a capability. This will imply some of the existing capability changes. On delivery of all of the increments, the capability has been replaced.

6.3.4 Emergent design versus enough design upfront

Some Agile approaches introduce the concept of emergent design. This means working on something that makes sense now and can add value into the organization; the experience from this will start to show what underlying design is required and/or exists. Refactoring then enables the earlier parts of the solution to be retrofitted to the emergent design. Such work is often defined as technical debt, as the project understands the work has to be done, but needs to fit it into the schedule of work (product backlog). Failure to do so will ultimately make solutions unmaintainable, and extend the time required to do work. Planning processes must consider this.

There is another problem with this approach if the project is to be at any scale. Different teams may start to create conflicting designs, and it may become difficult to introduce new members to teams, or new teams into the project, as the design may be implicitly understood but not well documented (although enough documentation should always be kept). Time may also be lost in the refactoring process.

A better approach is what is called ‘enough design upfront’ (EDUF). This examines the solution sufficiently to be able to identify key design considerations and constraints and to set a framework in which the teams can operate. Enough is done only to have a high-level understanding and to avoid a lot of rework in the future. The design is not fixed in stone, but can and will evolve. The evolution is always based on discussion and agreement, so that the technical consistency of the whole solution remains intact. The word ‘technical’ is used in the broadest sense, and not only as an IT term. There are many other advantages to EDUF. These include understanding how a solution can be partitioned to deliver value early and often, and identifying basic components that would need to be prioritized into early timeboxes. When new solutions are replacing existing ones, the design can help in determining how early releases can be integrated into the existing solution.

Figure 6.4 shows the difference between a city which has necessarily evolved over time (left), and a city which was designed based on some initial planning (right).

6.3.5 Regular releases

Regular releases are an important part of most Agile approaches. They ensure that the customer gets value as soon as possible. We have already discussed the considerations for planning releases with respect to customer disruption and operational services involvement. There are two other areas that need consideration.

Firstly, the release will be formed of new capabilities. These need to be introduced into the organization and that will require resource from the team. Activities requiring particular attention from the team include moving components into live environments, migration activities, changing business processes, training and early life support (an ITIL term). As well as resources from the team, this is likely to include other resources.

Secondly, as soon as a capability is introduced into an organization, it becomes an integral part of the business and its absence will cause significant disruption. If this is not the case, the original reasons for developing the capability will have been flawed, as it does not appear to add much value. This means that plans for supporting the capability need to be in place, including fixing bugs. This work often falls to the team that originally worked on the capability. In an incremental delivery environment, however, this team is already working on the next set of features.

It is therefore important that increment and timebox planning incorporates time and resource for these activities. For example, the bugs and enhancements that emerge from live use need to be incorporated into the product backlog, or equivalent, and prioritized along with the new features that are planned. The good news is that the amount of rework is normally much reduced, as the users have been involved throughout the development process.

6.4 Incorporating change

Agile places a very strong emphasis on embracing change. One of the four key statements in the Agile Manifesto is ‘Responding to change over following a plan’. We accept that change will happen, and that it should be easy to incorporate it into the projects and programmes we are running. There is, however, a big difference between accepting that change will happen and understanding how we can incorporate it. The situation becomes even more complex if we consider contractual agreements between different organizations. Each side is working to agreed timescales and budgets, and change can easily affect them. Merely stating that we have an environment of trust and empowerment will not solve this issue. Many customers have based contracts on receiving all specified features. Many suppliers have operated on a business model of bidding low to get the work, and making money from changes. Clearly, both suppliers and customers have to change their mindsets to think of incorporating change as a normal activity.

The customer has to accept that it may not get everything it originally wanted, but it will get a capability that meets its business needs for today, because the inevitable change that has occurred since the initial vision was formulated will have been incorporated.

The supplier has to accept that it will deliver capabilities that are relevant, robust and usable today, incorporating changes. Building trust and confidence and delivering a good solution will ensure that it gets follow-on work.

6.4.1 Initial planning for change

So how can we accommodate change without seriously compromising budgets and/or timescales? The answer comes in the way we look at the changeable parameters of the project. A traditional model tries to fix the features that will be produced and change time and/or resource if the need arises. Many Agile approaches turn this on its head. Time, cost and the level of quality are fixed and features are changed if necessary, as shown in the ‘iron triangle’ (Figure 6.5).

Some of the features are vital to the capability but others, although they add value, can be removed or changed. This implies that features can be prioritized. The MoSCoW prioritization technique (as defined in Table 6.1) can be useful.

If we estimate our project, based on the ‘must haves’ being 50–60% of the effort required, we will have:

• Catered for actual effort being up to double the estimated effort
• Provided a way of incorporating change by replacing original ‘could have’ or ‘should have’ requirements and replacing them with the newly required features.

To a large extent, as long as the changes maintain the ‘must haves’ within the 50% tolerance, the teams can deal with the changes without undue bureaucracy. However, changes should always be recorded; otherwise it becomes difficult later to determine exactly what was and was not included in the capability, and it can be difficult to understand the reasons for change. If changes are likely to jeopardize delivery of the ‘must haves’, procedures that are more formal will still be necessary, as discussed in Chapter 3.

A planning approach that accommodates change can be built into contracts, as it will adequately protect both sides from significant risk.