There is no point in leading the perfect teams and developing the product impeccably if it won’t provide value to the customer. The Agile Manifesto is clear that the customer and the business are critical elements of an Agile delivery, so it follows that managing the product and ensuring it will deliver value to the customer is extremely important.

Some requirements practices were introduced in Chapter 8, but this chapter goes into more depth to help you focus on managing the product, including identifying customers and stakeholders, understanding their needs and validating that those needs have been met.

Throughout this book we have emphasised the value and benefits Agile methods can bring. However, few Agile approaches discuss in much detail how to identify user needs and requirements in the first place, a well-formed backlog is just assumed to exist. Yet, without a clear view on the user need, we will struggle to maintain a focus on value.

In fact, it is often the case then even the customer isn’t clear on what they need. It is common for customers to describe the product or solution they think they need, but this solution focus can often mask their real needs. We must find ways to better understand those needs while still adhering to the Agile values and principles and avoiding big up-front requirements capture.

This chapter will uncover and share techniques to help you:

• understand customer needs;
• capture product ideas in roadmaps;
• break those ideas down into manageable work items;
• organise them in product and iteration backlogs.

UNDERSTANDING AND MANAGING STAKEHOLDERS

The statement above seems obvious, yet all too often we see products not meeting customer expectations and disappointment experienced by both the customer and the delivery team. There may be many reasons for this, but essentially it comes down to not properly understanding the customer or wider stakeholder needs or expectations.

First, it is useful to define what we mean by stakeholder. We define a stakeholder as anyone who has a vested interest in, or is affected by, the product.

A stakeholder, therefore, is wider than just the user and customer and can include many people with knowledge or interest in the product, such as:

• the person paying for the product, who may not be an end user;
• subject matter experts;
• managers of end users;
• external partners and suppliers;
• technology specialists and architects;
• security experts;
• legal advisors or lawyers;
• user experience designers;
• sales and marketing.

Each of these stakeholders needs to be understood, as well as the more obvious customer or user, as they may have an impact on whether our product is successful. If we only focus on the customer or end user, we could deliver a product that meets user expectations but may fail in other ways, such as not integrating with other systems, being too expensive to produce, failing legal or security compliance or addressing the wrong market segment.

For the rest of this chapter, we will focus on understanding and building a relationship with the customer who will benefit most from the product. However, all these techniques can, and should, be applied more widely to other stakeholders.

Challenges in understanding customer needs

We know that understanding your customer is critical to the success of the product, yet this is not always as easy as it should be. Some of the challenges we can face are discussed in Table 10.1.

There are several techniques that we can use to understand our customers and their needs. Some of our favourites are shown in Figure 10.1 and described in the following sections.

Context diagrams

Before delving into user stories or identifying user needs for a product, it is a good idea to first understand and agree the initial context of the product. This is a starting point for defining the scope, which is important for managing user expectations.

There will always be more potential work than you will have time or resource to accomplish, and customers are adept at identifying more once you begin. Thus, defining and agreeing an initial scope boundary is key to meeting product goals.

A context diagram shows the interactions between a product, or system, and the things outside that product/system; these interactions can be people, other systems or other organisations. It provides a solid foundation for exploring these interactions with the context of the product, and is a great starting point to explore user roles. An example of a context diagram, for our loyalty card programme, is in Figure 10.2.

Context diagrams are one way to understand and define a product context. They are also described as ‘boundaries and interactions’ in Diana Larsen and Ainsley Nies’s book, Liftoff.¹⁷⁵ Examining boundaries and interactions is important throughout Agile chartering – in purpose and alignment but especially in context, where understanding the external relationships and dependencies is critical to success. Exploring the context can identify (or confirm) where the team relies on others for an element of the solution. For example, teams often forget about internal stakeholders such as sales, marketing, legal or security whose involvement is essential to the product delivering value.

Empathy mapping

The empathy map¹⁷⁶ was designed by David Gray as part of a toolkit called Gamestorming.¹⁷⁷ It helps teams to obtain shared understanding and empathy for others, which particularly helps to improve customer experience. It also works well in conjunction with personas (described next). The empathy map contains seven key areas, as shown in Table 10.2.

Considering each of these elements helps us to ‘walk a mile in the customers’ shoes’ and can lead to more insightful analysis of requirements and better alignment of priorities with customer needs. Building empathy with your customer will also improve your relationship with them.

Personas and user roles

Once the product context is understood we can set about better understanding the customers of the product. Personas and user roles are two such techniques for this; however, we often see them being misused or amalgamated into one, so they are worth considering together.

Personas and user roles are complementary, yet distinctly different, techniques that both help us to understand our customer better. They can be used separately, but are most powerful together. The main distinctions are discussed in Table 10.3.

Figure 10.3 shows the relationship between role and persona and shows the differences. Creating personas will help to add value to our understanding of a role by helping us to consider the emotional and behavioural aspects that different types of user bring.

A common mistake is to extend personas to cover both the role and the persona. This is unnecessary, confuses the distinct value brought by the two perspectives and results in duplication where multiple personas all share the same role.

Personas can represent a range of users, not always a single user role, and a single user role can be taken by many personas. For example, within our loyalty programme we have a user role that is Loyalty Card Customers. They perform tasks such as view loyalty points, earn loyalty points, spend loyalty points. Some personas associated with the Loyalty Card Customer role include:

• elderly customer with no IT;
• tech-savvy customer;
• cost-conscious customer.

While they will all want to earn loyalty card points and view their current loyalty points, they may also have different needs determined by the persona. For instance, a teenage customer who is technology savvy may only want to engage through a mobile app, while an elderly customer without a smartphone may only want to interact by post or telephone.

Knowing the relative percentages of personas is important: if 15 per cent of customers don’t have or don’t want to use IT, we still need to consider their needs in our product development, but we may want to focus on the other 85 per cent first. Conversely, if the 15 per cent are higher spenders or more frequent shoppers, we may want to focus on them first.

Similarly, our cost-conscious customer persona could also work for the store and therefore sometimes be in the Cashier User role and interact with the system in a different way. Examples of a role description and a persona are shown in Figures 10.4 and 10.5.

One simple way to avoid confusing roles and personas is to ensure that each role has multiple personas associated with it. Having a 1:1 relationship is an anti-pattern that should be avoided, as shown in Figure 10.6.

If you are struggling to find more than one persona for a role, you may find it more useful to just have one role and use empathy mapping (discussed earlier in this chapter) against that role. Creating just one persona is probably creating more work than is necessary and doesn’t really add much value.

Customer journey mapping

Mapping the customer’s journey through your system will help you to understand their needs and identify possible pain points or new opportunities. A customer journey map is developed from the perspective of the customer and explores the path they take to achieve a goal or objective, such as ‘Register for loyalty card’. It includes emotions and feelings as well as functional steps, so can also help to feed into an empathy map.

The first step is to break the journey into small and simple steps such as ‘Find Registration Page’ or ‘Confirm Email Address’. Then, capture the experience of the customer while navigating these steps. This is usually done by observing the customer and asking them how they feel at each interaction, but you can also ask them to provide feedback through a questionnaire or interview. This technique will give the best results with real customers using the product for the first time. The information is captured in a map (see Figure 10.7) along with other observations, such as opportunities to improve that you have identified or additional feature ideas.

You will probably need to repeat this with different scenarios to get a complete picture. For instance, in our example we may want to have different journeys for people using a mobile device or a desktop computer, or people who use password managers and those who don’t. Some things to consider when creating a customer journey map include:

• Be clear of the goal to be achieved by the customer journey.
• Ensure you have defined personas with clear goals listed.
• Identify the steps and interactions you expect the customer to take to achieve their goal.
• Take the customer journey yourself.
• Observe the customer as they carry out the steps to achieve their goal, listing their emotions and experience at each step/interaction (e.g. happy, sad, need more info, not sure what to do).
• Let them make mistakes, and don’t correct them. Seeing where they went wrong can be very useful.
• Make improvement/changes as necessary and repeat the process until the customer journey is optimised.

Story mapping

Customer journey mapping is a great way to understand the external view the customer takes, but sometimes we need a little more structure to explore a problem in more depth. One of our favourite ways to do this is to apply Jeff Patton’s user story mapping approach.¹⁷⁸ User story mapping allows us to visualise product delivery in a way that promotes early delivery of value yet still retains the ability see the whole system and make changes easily.

A user story map is created on a wall, and organises the work into two dimensions. Across the top is the value stream: the types of changes, tasks or actions that need to occur to deliver the value that we are aiming for. This is the backbone. Then we brainstorm all the detailed tasks, stories, that we can implement. Figure 10.8 shows a snapshot of this process.

The creation of the backbone and stories is a powerful way to get a shared understanding across the team and customers of how the product will create value. When it represents the customer’s journey, it helps to build empathy between the team and the customers. It focuses the work on things that will impact the customer and provide them with value. At this point the underlying stories are not prioritised.

The next step is to organise the stories vertically by arranging them into horizontal rows, each representing one iteration or release of the product. Each row contains the specific things under each backbone element that need to happen to deliver value to the customer. Backbone elements are moved down if they are not essential to that particular iteration. Not every part of the backbone needs to be incremented at each iteration, for example the ‘Generate Insights from Trends’ column for our loyalty card system doesn’t have any effort until after several iterations. Figure 10.9 shows this.

All the items are represented by sticky notes and can be moved up or down the board easily. This promotes powerful conversations with customers about what is important in early releases and helps us to identify the minimum set of features necessary for the customer to get some value –the MVP we discussed in Chapter 7.

One nice aspect is that it allows us to visualise all the possible product features on one map. This means that stakeholders can find their ideas on the map and see where they are in relative priority to other features. This makes the likely order of delivery clearer and it’s easier to see why a particular story is where it is.

Being able to see their stories on the map gives comfort to stakeholders that their stories haven’t been forgotten. Because we get them involved in deciding their position, there are fewer problems with stakeholders trying to bump their stories into an earlier release.

Because the backbone represents the intrinsic value chain of the product, it changes less frequently, meaning we can be more certain of it. Conversely, the things that are likely to change – the priority of specific features and stories – are easy to change since we just move them up or down the map.

An initial user story map can be created in a workshop over a couple of hours and evolved as everyone’s understanding of the problem grows. Even though they can be detailed and quite large – in Jeff’s book, he gives some examples that represent years of work for quite a large team – they can still be created in a day or two, even for large and complex products. We have also used them very effectively for small products.

Customer discovery and Lean Startup

Agile teams are highly customer-centric, so techniques that place the customer at the centre are a natural fit. ‘Customer discovery’ is a term used by entrepreneur Steve Blank,¹⁸¹ and is exemplified by his mantra ‘get out of the building’ – customer discovery involves finding out what your customers need by asking them, and you can’t do that by sitting in your office.

You might find out what they want by better understanding their needs or pain points, or you might find out whether a product or idea of yours would be attractive to them. In either case, you start with a hypothesis, conduct some experiments or tests and evaluate the results.

Ideally, this process is quick and cheap, so it is important to find the smallest possible experiment that will give you some useful information. By using real customers, we get data that we can trust far more than our own guesses. That’s why it is important to get out of the building. This type of information is validated learning.

The Strategyzer¹⁸³ series of books and models from Alex Osterwalder are a great source of ideas for customer discovery and identifying or developing your product. The Business Model Canvas, Value Proposition Canvas and Customer Gains and Pains can help with aligning the team with a common purpose and help to identify your high-level product goals and vision.

ROADMAPS IN AGILE DELIVERY

An Agile iteration is usually short; one to three weeks is common. This allows frequent delivery and presents a timeframe within which drastic change is unlikely, but it makes longer-term planning hard. Most real-world products will take several months, or even years, to complete.

Despite wanting to keep our options open and respond to change, we often need to have a longer-term plan or view on the development of our product. This may be required to secure funding, align with other initiatives or to inform things like marketing plans or hardware procurement. More often, however, it is required on the whim of a senior stakeholder who thinks having a longer-term plan written down will make it more likely to be true. The challenge for Agile teams is to provide enough clarity on longer-term delivery without resorting to detailed planning that is likely to be wrong.

While storing this detail in one place, such as a backlog (see Managing the Backlog later in this chapter), may seem logical, it can become unwieldy over time and prone to becoming too detailed. Instead, we create a separate artefact often called a ‘roadmap’. The roadmap makes it easy to distinguish between short-term and long-term needs and makes the work more manageable.

We can create a trustworthy roadmap by abstracting the detail to a level we can be sure is unlikely to change. The items are bigger, but we don’t need to go into detail. This could be in the form of a product vision or a roadmap of large features, for instance we may have a roadmap item to implement personalised offers to the customer. We don’t need to decide which kinds of offers they will be, what customer data are used for the personalisation or how they are distributed, all that detail can be left until later.

A roadmap isn’t a plan, it’s just our current ideas of what we think we may do. To help stop people thinking of it as a plan, it’s a good idea not to have any dates on it. Instead, structure the roadmap in three sections: Now, Next, Later, as shown in Table 10.4.

In this way, stakeholders can understand the overall scope of the product, retain the ability to change priority, and add or remove features without worrying about replanning. The Later section can be prioritised to indicate the rough order of delivery, and we can estimate the relative sizes to help understand the overall size of the delivery. However, you don’t have to do that; a simple list of things for the future is good enough.

Another approach we can take is to set bigger goals than can be delivered in a single iteration. These could be on a longer cadence (perhaps a couple of months) or be a variable length depending on the goal itself. These goals could represent the set of capabilities that allow a formal release or the minimum set of features for a particular set of users. We would still have goals in each iteration, but they would contribute towards the higher-level goal (see Figure 10.10).

We want our roadmap to have just enough detail to be useful and not be so intricate or detailed that we are reluctant to change it. The ‘Endowment Effect’ (see Chapter 4) and the ‘Sunk Costs Fallacy’¹⁸⁴ make us less likely to want to change something we have invested time and effort into. Making these artefacts simple to change and easy to replace will help to avoid that.

AGILE REQUIREMENTS

The word ‘requirement’ has been around for decades and is commonly used in many traditional and Agile teams. It is used to describe a feature or need from a stakeholder that may form part of a solution or system. Some requirements represent the usability or functional needs of the solution, whereas others will impose constraints on the solution, such as an architectural constraint, or non-functional requirements, such as a performance target.

In more traditional approaches formal requirements are elicited and documented up front, in advance of development work. They are often validated through formal review or sign-off processes to ensure they are correct before development begins. As discussed in Chapter 4, this creates a form of contract, sometimes even a legally binding contract. This approach is shown Figure 10.11.

The problem with traditional requirement approaches

There are many problems associated with conducting requirements work in this way. Some of these are included in Table 10.5.

An Agile requirement approach

So, what is the alternative? In Agile delivery the requirements should evolve over time. Requirements artefacts are lightweight and produced at the point they are needed and not in advance. There is high emphasis on collaboration – those that build the solution must speak to those who will use it if they are to fully understand what they are trying to achieve. Lastly, the requirements do not get ‘signed off’ in an Agile approach. As more value is placed in working software than comprehensive documentation, it is the working software that is signed off.

As explained in Chapter 6, Agile teams deliver value incrementally and iteratively, delivering increments of value early and frequently. They set goals that are valuable to the customer and deliver them quickly. Since small goals can be completed quicker than large goals, Agile teams focus on small units of value. This approach is shown in Figure 10.12, and the advantages of an Agile approach include those listed in Table 10.6.

Adopting an Agile requirements approach can be challenging, since the alternative is so prevalent in organisations and project management culture. Requirements skills are typically restricted to specific roles such as requirements engineers, business analysts or systems engineers, yet the people most affected by the quality of requirements are the users and developers of the solution.

The most successful teams bring together technical and analysis skills coupled with business knowledge. The developers can bring solution-centric ideas and innovations. After all, the end users may not be experienced software developers and therefore cannot know the art of the possible. Equally, the user view is important, perhaps to temper a deeply technical solution that may be far more complicated than the user actually needs.

REQUIREMENT TECHNIQUES

It is important that we choose techniques that enable the user needs to be elicited just enough and just in time. Just enough so that there are enough work items in the backlog for the team to elaborate for development in the next iteration; just in time to ensure that we are not eliciting and detailing requirements that may not be important, therefore creating requirement inventory and risking the detail becoming out of date.

There are many methods that assist with articulating and navigating requirements within an Agile context. Some of the more popular techniques are shown in Figure 10.13 and described in this section. For a more thorough discussion, see the BCS book, Agile and Business Analysis.¹⁸⁶

User stories

A user story is a short description of value and often follows the following format, pioneered by Connextra in 2001,¹⁸⁷ though it doesn’t have to:

As a <role> I can <capability> so that <receive benefit>

For example:

As a Loyalty Customer I can view my points balance on my phone so that I can decide whether to redeem my points in store.

This format helps teams to focus on who the benefit is for and why they want it. It lends itself to short, specific requirements, helping teams to keep their batch size small. Traditionally, a user story is written on an index card, which also helps to limit size by limiting the amount of space there is.

The small, simple nature of a user story also requires the team to talk to the customer before they can implement it. Alistair Cockburn first coined the phrase ‘A user story is a promise of a conversation’,¹⁸⁸ which is an excellent way to help us focus on the customer collaboration side of the Agile Manifesto rather than the contract negotiation side. User stories must also have acceptance criteria that describe how the team will know that the story is complete.

The small scope of a user story means that there may be many stories in a product compared to other approaches for documenting requirements, such as use cases (discussed later in this chapter). Where a use case would describe a higher-level requirement with multiple steps and paths, a user story could represent a single scenario or path. For example, in the use case ‘Manage Points’ there would be many additional tasks that a user may want to do, not just view their points balance. Each of these could be an independent user story, prioritised and estimated independently. User stories are also limited to the user’s needs – they don’t imply any solution.

For more information on user stories, see Mike Cohn’s book User Stories Applied.¹⁸⁹

Epics and splitting them into smaller stories

User stories are intended to be small enough to be completed within one iteration. When they are too big, they are called an epic.¹⁹⁰ Other terms used for large ‘stories’ include feature, capability, goal.

When we first come up with things that our product can do, it is common for many of these ideas to be quite big, even if they can be written in the user story style. For example, one of the later stories in the loyalty card example might be:

As a store manager, I can use the data to help me decide how to manage in-store promotions so that our promotions can be more effective.

That story probably won’t be able to be done in one iteration, so it’s not a story, it’s an epic.

When an epic is prioritised to the top of the Product Backlog, we will need to split it into multiple, smaller stories and then reprioritise them before we can choose what can be done in the next iteration. There are several ways to do this, some that result in the epic being replaced entirely by lots of smaller stories, and others where small, achievable stories are sliced off the epic but the rest remains as an epic, to have further stories sliced from it later. Some useful techniques on splitting stories, epics or goals are shown in Table 10.7.

Table 10.7 shows how relatively small stories can be split into even smaller stories. We also need to split things that are even bigger than epics. Often, there are huge chunks of functionality that we capture but don’t want to break down until later. For example, the loyalty card system has identified a business need for data analysis including using artificial intelligence to generate insights into customer spending.

That’s huge and can be broken down in different ways. In that example, a good tactic is to treat it like an iceberg. Identify a small goal that is part of it and slice it off – for example, Offer Penetration: how many app users view the offers in their AG Stores app when they get a notification that a new offer is available? This new story is ‘calved’ from the Data Analysis iceberg. The rest of the iceberg stays where it is without any further decomposition necessary until data analysis becomes a priority again.

Some other tactics for dealing with very large roadmap or backlog items include:

• Scenario flow (basic flow, alternative flow, exceptions) – Often there are many ways that a user can achieve a result. There are also often several ways their attempt can fail. This can result in there being many different possible permutations. Each of these can be split out and implemented separately.¹⁹¹ Use cases (described later) provide a nice structure where identifying these flows is straightforward.
• User type/persona – Different users or personas can give us a different view of the system from a user perspective, so can be used to help us take a generic story and spit it into multiple stories for each perspective. For example, different persons may access a product or service in different ways.
• Good-Better-Best – Jeff Patton introduced this concept in his book User Story Mapping.¹⁹² For each story/epic consider what needs to be in place to make it just good enough, what more can be added to make it better and what is the best it can be. This would create at least three stories and help to focus on the most important things first and prevent over-engineering solutions.

Examples are given in Table 10.8.

Each of the above examples of Good-Better-Best are stories in their own right and all achieve the goal of providing customers with an update on their loyalty card points. However, how this can be achieved varies and enables us to consider different initial options for the MVP.

This technique is particularly useful if you are trying to deliver more than one story in an iteration. Split them down into Good-Better-Best and use MoSCoW to prioritise them. Ensure that ‘Must’ stories are no more than 60 per cent of your available capacity.

In any elaboration or further analysis of requirements it is important to bear the simplicity principle in mind: maximise the work not done. Only split out what you need to in order to deliver the maximum value to the customer. Nothing more.

The three Cs

Ron Jeffries introduced the term the ‘3Cs’ in Essential XP.¹⁹³ This describes the three critical elements of a user story: card, conversation and confirmation (see Table 10.9).

Use cases and Use-Case 2.0

Use cases were developed by Ivar Jacobson and presented at the OOPSLA conference in 1987.¹⁹⁴ They became a very popular way to describe and model requirements of both large and small systems. They are integral to the Unified Modelling Language (UML) and a core component of the Rational Unified Process (RUP). They evolved into Use-Case 2.0¹⁹⁵ to better support Agile delivery and are still a popular and powerful requirements tool today.

Use cases are a lightweight, iterative way to describe what a system is going to do and, by intentional omission, what it is not going to do. They tend to describe larger, more comprehensive pieces of functionality than user stories, but the two approaches often successfully complement one another.

There are six key principles that underpin Use-Case 2.0, as shown in Table 10.10.

Use cases are iterative and progress through a series of states, as shown in Figure 10.14.

Teams often use a template for use cases with sections for narrative descriptions of the use case in increasing detail. For instance, there are often sections for brief description, main path, pre-conditions, post-conditions, alternate flows, exceptions and so on. Each of these is an opportunity to describe the use case in more detail, each section building on the others. Ideally, this detail is added in business-friendly language such as a story, but use cases can also form part of detailed UML models where multiplicity, inheritance and other technical details can be documented.

Despite their potential for complexity, use cases can be extremely valuable in their simplest state – with just the overall goal described. This is often enough to prioritise the system development, create an initial backlog and begin identifying the highest priority use case slices or user stories. Low priority use cases may not need further elaboration until much later in development, perhaps never if their priority changes and they are not required.

At this high level, a use case diagram (see Figure 10.15) can be a very useful way to express the scope of the system and keep stakeholders informed. These are quick to create and provide a system- or subsystem-wide scope that can help communication and planning. More complex modelling of use cases can elicit more detail but remember the simplicity principle.

Non-functional requirements

It is easy to overlook non-functional requirements when so much effort is expended on understanding and managing the customer needs and expectations. However, non-functionals, or ‘NFRs’, are just as critical to the product success as functionality and can cost more in the long term if not considered. The difference between functional and non-functional requirements is defined in Figure 10.16.

In many ways non-functional requirements define the quality characteristics of the solution. They define things such as performance, reliability, security and the service level agreements that document how the solution will be maintained and supported through its lifetime. We categorise them in two ways, as in Table 10.11.

Both types of non-functional requirements need to be documented and managed. One strategy that we recommend is as follows:

• Global non-functional requirements are formal requirements and should be managed in a formal, version-controlled system. In some cases (such as pharmaceuticals, finance or defence industries), they may be subject to audit and compliance, which may require a formal, external process to be followed. They should be considered and documented early in the product life cycle as they can describe quality criteria that can influence architecture decisions.
• Associated non-functionals relate to functionality within the system, so are directly related to groups of stories. Rather than copy them into every relevant story’s acceptance criteria, it makes more sense to write them once – perhaps in a behaviour driven development (BDD) language such as Gherkin (discussed later) – and refer to them in each relevant story. It can be convenient to use the same document as the global non-functionals.

For both types of non-functional requirement it is important not to forget about them, which can happen when details are hidden away in documents. To overcome this, add them as sticky notes on your backlog or Agile board. This helps to make them more visible, as stories and other backlog items are considered each iteration. They can then be included in the acceptance criteria for the story – the non-functional requirements themselves are never ‘completed’ and do not go through the same workflow as functional requirements do.

Behaviour driven development and acceptance criteria

BDD was introduced in the 2000s by Dan North.¹⁹⁶ It emerged in response to TDD (see Chapter 7) as a way to help prevent misunderstanding between development and testing in Agile teams by creating a language for testing business value and behaviour at the acceptance level. BDD aims to provide a shared understanding between the different perspectives of the developers, end users and testers by ensuring that only software that meets the business need is built.

The language used for BDD is called Gherkin. It is based on the user story format of ‘As a … I want … so that …’, but uses the construct given-when-then, which is defined in Table 10.12. Figure 10.17 is an example of a BDD scenario for our loyalty programme.

The scenario can contain more detail, which can help when using automated testing systems. Figure 10.18 shows a more detailed version of this scenario with actual values.

The relationship between BDD and user stories does not stop at the similarity in format used. In fact, every user story can have multiple BDD scenarios to test the behaviour and value of the story. In this way, they can become the acceptance criteria and tests for the story. For example:

As a Customer I want to register for a loyalty card via the website so I can earn and redeem loyalty card points.

The BDD scenarios that may be used to test this story include the examples shown in Figures 10.19 and 10.20.

It is easy to see how BDD can be used to write tests or meet the acceptance criteria as part of the user story confirmation. However, it should be noted that BDD tests should only be written once a story is in an iteration. They are developed by the Agile team in conjunction with the end user or subject matter expert. Writing BDD scenarios for stories that are not yet prioritised into an iteration can be an incredible waste of time. It also usually means that the development team has not been involved in writing them, increasing the risk that they will be wrong or misunderstood.

BDD also makes it easy to implement TDD (see Chapter 7). The first ‘failing test’ can be the BDD for the expected behaviour. Then, once that works properly, additional BDD scenarios can be added as new failing tests to be implemented.

Definitions of ready and done

The definition of ready (DoR) and definition of done (DoD) are mechanisms designed to reinforce transparency and assure built-in quality (see Table 10.13). They provide an agreed set of rules that smooths the transition of the work item into the development team and onto the user, and ensures that expectations of the work item to be planned, developed and completed are set and clear for the whole team.

The DoR ensures that once the team starts working on the item there should be nothing external that stops them completing it. They have all the information, data, permissions, access to customers and so on. that they need, including from wider stakeholders (users, auditors, marketing, security, etc.).

The DoD ensures that once the team thinks they are complete there is nothing to prevent the product being used by the real customer to meet their real goals. It will be tested, safe and legal to use, secure, in the right environment, documented enough, using the right branding, marketed, accessible and so on.

The definitions are owned by the whole team (including the customer or Product Owner), which makes everyone more likely to adhere to them. They evolve as the development proceeds and should push increased quality standards over time. This can make them an indicator of the maturity of the Agile team.

The DoR and DoD can be seen to represent the agreement of the boundary between the development team and customer. The DoR represents what the customer will do in order to prepare the work item ready for the team to start work on it straight away. The DoD represents what the team will to ensure that the work item can be used by the customer straight away.

They can also be seen as setting entry (DoR) and exit (DoD) criteria for work items, although they must not be confused with stage gates. Their purpose is to aid flow of work, not impede it. Examples of DoR and DoD for a work item such as a user story are shown in Table 10.14.

Both DoR and DoD can and should be applied at all levels of abstraction that the team use, for example a work item, feature and release levels. Some aspects may only be relevant at higher levels, but, more importantly, we need to test and accept higher-level elements independently. We can’t assume that a story is done just because its low-level work items are done. Similarly, headline features or capabilities in a product should be tested explicitly, not just assumed to work because the underlying stories are finished.

MANAGING THE BACKLOG

A critically important Agile practice is the creation and maintenance of one or more backlogs. The Product Backlog is the main one and it contains the things that we think will be necessary to deliver the value to the customer.

In some methods, there is also a backlog for the current iteration. This backlog contains a lower level of detail that helps the team to ensure that they know what they need to do to meet the iteration goals. There may also be higher-level, strategic backlogs with portfolio or programme level items. Generally speaking, the fewer backlogs the better.

Sometimes, there may not be an obvious backlog at all. The minimum necessary, assuming the team isn’t working on the final story, is that the team knows what the next highest priority work is and that it is elaborated enough for them to complete in their next iteration. Whether you have one or multiple backlogs, and whether they are called a backlog or not, the product’s queue of future requirements needs to be managed and maintained. In simple terms, the backlog is a prioritised list of work items that contains the most important work items at the top and less important work items at the bottom (see Figure 10.21).

In Mike Cohn’s book, User Stories Applied,¹⁹⁹ he refers to the backlog as an iceberg because the work items at the bottom are less developed than the work items at the top (see Figure 10.22). Only the work items at the top, above the water line, will be developed to the point at which they can be accepted into an iteration. Similarly, the items below the water line are more likely to be big stories (i.e. epics) and will need further work and analysis to break them down.

Prioritising the backlog

The iceberg metaphor is also a helpful prioritisation technique, since it provides a simple, two-state priority system: above water or below water. Items above water are prioritised for delivery and should be prepared so that the team can pull them into their next iteration. If an item is below water, we should not elaborate it further until it rises above the water. As the above-water items deplete, a ‘backlog refinement’ session will assess the items below and select which ones should be prioritised higher.

You will note that Figure 10.22 also shows small stories below the water line. This is not unusual; sometimes, even low priority stories are small. However, this also occurs when an epic is split into three or four smaller stories, but only one or two of those stories are deemed important enough to be above the water line. The remaining stories then take their rightful place in the prioritisation order, which may be anywhere depending on importance.

This approach is also useful for backlogs with more detailed prioritisation systems. Many backlogs employ a unique priority system, as shown in Figure 10.23. The position of the item determines its priority. During backlog refinement we apply a similar approach to splitting epics as before. An epic may occupy a unique priority, but once split each story that emerges is independently prioritised.

The original epic could have been near the top of the backlog, but needed splitting since it was too big for the next iteration. Despite this, some of the stories could end up quite low priority, for example we may have an epic that allows customers to pay through their app. We wish to prioritise this to test its popularity. We split the epic into smaller stories and prioritise the easiest one that allows payment. The others will only be prioritised if the experiment is a success, until then they are low priority.

Chapter 8 describes several prioritisation techniques that we can apply to help us prioritise and split stories or epics on the backlog. Whichever technique(s) you use, remember that the most important part of a backlog is the top. At the top is the most valuable work the team can do next, of a size and level of detail that they can start immediately. Any work elaborating work items beyond that is potentially wasteful.

Judging how much work needs to be in this state means knowing something about how much work the team can do in each iteration. This is dependent on several factors, including team capacity, complexity of the work and size of the work. If the complexity of the work items is very variable, then including some estimate of complexity alongside size and value can be helpful when prioritising the work. However, remember the simplicity principle: only do this if you need to, and even then do ‘just enough’.

Backlog refinement

The order of the backlog is likely to change over time and so needs careful and constant management. We refer to this as ‘backlog refinement’ and it is an activity that should be carried out by the whole team as and when necessary, as follows:

• when a change happens that affects the work to be done next;
• when the number of ‘ready’ backlog items starts to run out;
• when a refinement event is scheduled;
• when the backlog gets too big.

Managing changes to the backlog in this way ensures constant review of the backlog so that it is always up to date but minimises unnecessary work. The fact an item is below the water line should be an indicator that there is little value in doing any work on it at the moment.

Managing a large backlog is an anti-pattern as it indicates that more customer needs and work items have been elicited and documented than are necessary. This results in effort having been spent organising and managing work items that may never find their way into a working solution.

To avoid this, and to keep the backlog lean, it is good practice to regularly remove items from the backlog. Some criteria you may want to apply when considering whether items should be removed from the backlog are:

• The work item has been in the backlog for more than six months and never been deemed important. (If you are wrong, it will soon get noticed and it will return.)
• The work item does not meet feasibility criteria.
• The work item is not aligned with the product/service vision.

SUMMARY

The backlog is where a customer need begins its journey to fulfilment and careful management, refinement and prioritisation will help you to ensure that the most valuable customer needs are always ready to be implemented next.

The backlog is also an important vector to good stakeholder engagement. Using innovative or creative ways to capture a backlog, such as user story mapping, can help to bring the product to life for stakeholders. However, we must also be careful to avoid setting unrealistic expectations or allowing stakeholders to view backlog items as commitments. Avoiding dates, maintaining short backlogs and regular refinement can help with this.

There are a great many techniques available to help with managing the backlog, not least because techniques from product innovation, prototyping and traditional requirements engineering can be used alongside Agile techniques.

Understanding and applying a number of these practices, including knowing when not to apply them, will help you to understand and document the product scope, customer needs and customer interactions with the product. They can allow you to create a backlog of small, value-centric goals that contribute towards the overall product through increments that customers find valuable. Achieving this is the domain of successful product and backlog management.

This chapter has just scratched the surface of the plethora of techniques available in this topic. We don’t expect all team members to be adept or experienced in these techniques as many of them can take years to hone. There may be specialists such as business analysts, product managers, user researchers or customer experience designers who may be available to help or coach.

174 Cohn, M. (2004) User Stories Applied: For Agile Software Development. Addison-Wesley, Boston, MA.

175 Larsen, D. and Nies, A. (2016) Liftoff: Start and Sustain Successful Agile Teams (2nd edn). Pragmatic Bookshelf, Dallas, TX.

176 Gray, D. (2017) Updated empathy map canvas. The XPlane Collection, 15 June. https://medium.com/the-xplane-collection/updated-
empathy-map-canvas-46df22df3c8a.

177 Gray, D., Brown, S. and Macanufo, J. (2010) Gamestorming: A Playbook for Innovators, Rulebreakers, and Changemakers. O’Reilly Media, Sebastopol, CA. See also: https://gamestorming.com.

178 Patton, J. (2014) User Story Mapping: Discover the Whole Story, Build the Right Product. O’Reilly Media, Sebastopol, CA.

179 Adzik, G. (2012) Impact Mapping: Making a Big Impact with Software Products and Projects. Provoking Thoughts, Woking, UK. https://www.impactmapping.org.

180 https://learnwardleymapping.com.

181 https://steveblank.com/tools-and-blogs-for-entrepreneurs/.

182 Ries, E. (2011) The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. Crown Publishing Group, New York.

183 https://www.strategyzer.com/books.

184 ‘The general tendency for people to make decisions to continue a behaviour or endeavour based on previously invested resources (time, money or effort) rather than future value’: Arkes, H. R. and Blumer, C. (1985) The psychology of sunk costs. Organ. Behav. Hum. Dec., 35, 124–140.

185 Barnes, M. (1988) Construction project management. Project Man., 6 (2), 69–79.

186 Girvan, L. and Paul, D. (2017) Agile and Business Analysis: Practical Guidance for IT Professionals. BCS, Swindon.

187 https://agilecoach.typepad.com/agile-coaching/2006/12/
as-a-coach-i-want-a-story-template-so-that-
people-ask-questions.html.

188 Alistair Cockburn coined this phrase during a visit to the Chrysler C3 project team in Detroit in 1998. It is referenced in an article on his website that is now unavailable but linked in his tweet: https://twitter.com/totheralistair/status/
897894659544031232.

189 Cohn, M. (2004) User Stories Applied: For Agile Software Development. Addison-Wesley, Boston, MA.

190 The word ‘epic’ is also used with other meanings, including in some Agile methods. We find this meaning most useful and use it in this book.

191 See Behaviour driven development (later in this chapter) and Test driven development (Chapter 8) for examples where this would be useful.

192 Patton, J. (2014) User Story Mapping: Discover the Whole Story, Build the Right Product. O’Reilly Media, Sebastopol, CA.

193 https://ronjeffries.com/xprog/articles/expcardcon
versationconfirmation/.

194 The Object Oriented Programming Systems, Languages and Applications conference. The conference would see the first public presentation of Scrum eight years later.

195 https://www.ivarjacobson.com/sites/default/files/field_iji_
file/article/use-case_2_0_jan11.pdf.

196 http://dannorth.net/introducing-bdd/.

197 Deming, W. E. (2000) Out of the Crisis. MIT Press, Cambridge, MA.

198 The INVEST mnemonic was coined by Bill Wake in 2003. https://xp123.com/articles/invest-in-good-
stories-and-smart-tasks/. It is sometimes adapted, for instance by replacing ‘small’ with ‘small enough’.

199 Cohn, M. (2004) User Stories Applied: For Agile Software Development. Addison-Wesley, Boston, MA.