Market and User Research

The foundation of customer centricity and design thinking consists of market and user research, which creates actionable insights into the problems customers face and the solution’s functional and operational requirements. Market research tends to drive strategy (who we are serving), while user research primarily drives design (how we meet their needs) (Figure 7-3).

Research activities occur continually and are supported through exploration in the CDP, product data analytics, and various feedback loops. Learning gained during market and user research also defines the solution context—the operational environment for a solution—which provides a basic understanding of requirements, usage, installation, operation, and support of the solution itself.

Conducting market research also helps determine the nature of the solution context. This context is primarily determined by whether the product is: 1) a general solution intended to be used by a significant group of customers or 2) a custom-built solution that is built and designed for a specific customer.

Understanding the solution context identifies external constraints that are often outside the organization’s control. Some aspects of solution context are variable (undecided or negotiable), and some are fixed (decided), and finding ways to manage this balance is crucial to value delivery. It impacts development priorities, and solution intent such as features, and Non-Functional Requirements (NFRs).

Identify the Personas, Problems, and Goals

Supported by market research, the next critical aspect of design thinking is to understand who will benefit from the product’s design. This information is captured by establishing personas, fictional characters (Figure 7-4) that represent different customer types that will similarly use the product. Personas help understand and empathize with the end users’ problems, experiences, behaviors, and goals.

Refine Personas and Establish Empathy

To further enhance the development of desirable solutions, customer-centric enterprises use empathy throughout the design process. Empathy maps (Figure 7-5) are design thinking tools that help teams imagine what a specific customer is thinking, feeling, hearing, and seeing as they do their daily jobs and use the product. The higher the degree of empathy that a team has for its customers, the more likely the team will be able to design a desirable solution. In turn, empathy maps help refine the personas.

Customer Journey Maps

Customer journey maps identify the process that a person goes through to accomplish a goal⁵ (Figure 7-6). They illustrate the experiences that customers have as they navigate a product from first engagement to achieving their objectives and thereby establish a positive and long-term relationship with the brand.

5. https://www.nngroup.com/articles/journey-mapping-101/

Story Maps

Story maps are an approach to organizing and prioritizing stories (Figure 7-7). They make the users’ workflow explicit and visible and show the relationships between user activities and the solution’s features and stories required to implement them. Story maps also help prioritize a group of related stories and ensure a conceptually complete set of system behavior is released together.⁶

6. https://www.jpattonassociates.com/storymappingslides/

Improving Design Feedback Through Prototypes

Prototyping creates functional models that provide initial validation of how a solution will potentially address the problem to be solved. They can be anything from paper drawings or mockups to a fully functioning aspect of the solution.

Prototyping helps the team clarify their understanding of the problem and reduces the risk of solution development. These mockups or models can be used for getting fast feedback or gaining clarity of the requirements for the desired feature or solution and new intellectual property and patent filing.

To gain actionable feedback, teams should strive to leverage the lowest-cost, fastest form of prototyping that best suits the learning in each situation.

Prioritizing the Program Backlog

Product Management has the primary responsibility for developing and maintaining the backlog and for making decisions regarding the sequence in which to implement features. SAFe applies a comprehensive model called Weighted Shortest Job First (WSJF) to prioritize work based on the economics of product development flow.⁷ WSJF is calculated by dividing the Cost of Delay (CoD) of a job by the duration. Jobs that can deliver the most value (or CoD) in the shortest period are typically selected first for implementation.

7. Don Reinertsen, Principles of Product Development Flow: Second Generation Lean Product Development (Celeritas Publishing, 2009).

In SAFe, the ‘jobs’ are the features in the backlog. Since it’s rarely possible to determine their duration before they are planned for implementation, SAFe typically uses relative job size as a proxy for the duration. Relative proxy measures are also applied for the CoD based on three components, as shown in Figure 7-10.

Using a simple table to compare jobs, WSJF is calculated for each feature (Figure 7-11). Unless there are sequencing dependencies between the jobs, the features with the highest scores are implemented first.

Each feature is estimated relative to the others for each of the three components of CoD and job size. The smallest item in each column is set to ‘one,’ and the others in that same column are estimated relative to that one. The CoD is the sum of the first three attributes for each item. WSJF is calculated as CoD divided by job size. The job with the highest WSJF is the next most important one to do.

PI Planning

PI planning is a cadence-based event that serves as the heartbeat of the ART, aligning all the teams on the ART to a shared mission and vision (Figure 7-13). PI planning is held face-to-face whenever possible. For geographically distributed ARTs, PI planning may occur at multiple locations simultaneously by maintaining constant audio and video communication between all sites. In some circumstances, like the COVID-19 crisis that is ongoing at the time of this writing, PI planning is fully distributed. The advanced topic article “Distributed PI Planning with SAFe”⁸ provides additional guidance and considerations for successfully managing this scenario.

8. https://www.scaledagileframework.com/distributed-pi-planning/

Facilitated by the Release Train Engineer (RTE), PI planning includes all members of the ART. It typically takes place over two days and occurs within the IP iteration, which avoids affecting the schedule and capacity of other iterations. PI planning is essential to SAFe: if you are not doing it, you are not doing SAFe.

Business Benefits of PI Planning

PI planning delivers many business benefits, including:

1. Aligning development to the business context and goals, vision, and team and ART PI objectives

2. Building the social network the ART depends upon

3. Identifying dependencies and fostering cross-team and cross-ART collaboration

4. Providing the opportunity for ‘just in time’ and ‘just the right amount’ of requirements, design, architecture, and user experience guidance

5. Matching demand to capacity and eliminating excess Work in Process (WIP)

6. Faster decision-making. All the relevant parties are focused on the same objectives and can consider and make necessary tradeoffs in real time

Inputs and Outputs of PI Planning

Inputs to PI planning include the business context, roadmap and vision, and the top ten features of the program backlog. Outputs of PI planning include the following:

• Committed PI objectives . A set of SMART⁹ objectives that are created by each team with a business value assigned by the Business Owners.

• Program board. It highlights feature delivery dates, dependencies between teams, and relevant milestones.

9. SMART is an acronym for Specific, Measurable, Achievable, Realistic, and Time-bound.

Conducting PI Planning

The RTE facilitates PI planning, and event attendees include Business Owners, Product Management, Agile teams, System Architecture and Engineering, the System Team, and other stakeholders, all of whom must be well prepared. The active participation of Business Owners in this event provides an essential guardrail for prioritization of business value. Figure 7-14 illustrates a typical standard two-day PI planning agenda.

Day 1 Agenda

• Business context. A Business Owner or senior executive describes the current state of the business, shares the portfolio vision, and presents a perspective on how effectively existing solutions are addressing current customer needs.

• Product vision . Product Management presents the current vision (typically represented by the next top 10 upcoming features) and highlights any changes from the previous PI planning meeting, as well as any upcoming milestones.

• Architecture vision and development practices. System Architecture and Engineering (typically a CTO, or Enterprise Architect) presents the vision for architectural changes and a senior development manager may introduce new or revised Agile development practices for the upcoming PI (e.g., such as introducing test-driven development or adjusting the CI/CD pipeline).

• Planning context and lunch. The RTE presents the planning process and expected outcomes of the event.

• Team breakouts #1. In the first breakout, teams estimate their capacity for each iteration and identify the backlog items they will likely need to realize the features. Each team creates their draft plans, visible to all, iteration by iteration.

During PI planning a program board (Figure 7-15) is used to visualize and track dependencies and to identify opportunities to eliminate or reduce them.

• Draft plan review. During the timeboxed draft plan review, teams present key planning outputs, which include capacity and load, draft PI objectives, potential risks, and dependencies. Business Owners, Product Management, and other teams and stakeholders review and provide input.

• Management review and problem-solving. It’s almost certain that the draft plans have identified challenges associated with scope, people and resource constraints, and dependencies. During the problem-solving meeting, management may negotiate scope changes and resolve other problems by agreeing to various planning adjustments.

Day 2 Agenda

• Planning adjustments. The next day begins with management presenting any changes to prioritization, planning scope, people, and resources.

• Team breakouts #2. Next, teams continue planning based on their agenda from the previous day, making the appropriate adjustments. They finalize their objectives for the PI, to which the Business Owners assign business value, as shown in Figure 7-16.

  

• Final plan review and lunch. During this session, all teams present their plans to the group. The team then asks the Business Owners if the plan is acceptable. If the Business Owners have concerns, teams are given the opportunity to adjust the plan as needed to address the issues identified. The team then presents their revised plan.

• Program risks. During planning, teams have identified program risks and impediments that could impact their ability to meet their objectives. These are addressed in front of the whole train and categorized into one of the following ROAM categories:

  • – Resolved—The teams agree that the risk is no longer a concern.

  • – Owned—Someone on the train takes ownership of the risk since it cannot be resolved during PI planning.

  • – Accepted—Some risks are just facts or potential problems that must be understood and accepted.

  • – Mitigated—Teams identify a plan to reduce the impact of the risk.

• Confidence vote. Once program risks have been addressed, teams vote on their confidence in meeting their team PI objectives. Each team conducts a ‘fist of five’ confidence vote. If the average is three fingers or above, then management should accept the plan. Any person voting two or fewer should be given an opportunity to voice their concerns. This might add to the list of risks, require some re-planning, or simply be informative. Once each team has voted the process is repeated for the entire ART with everyone expressing their confidence in the collective plan.

• Plan rework. If necessary, teams rework their plans until a high confidence level can be reached. This is one occasion where alignment and commitment are valued more highly than adhering to a timebox.

• Planning retrospective and moving forward. Finally, the RTE leads a brief retrospective for the PI planning event to capture what went well, what didn’t, and what can be done better next time.

Over the course of the program increment, the ART proceeds to execute the PI, track progress, and adjust plans as needed to adapt as new learning occurs. Execution of the PI begins with all the teams conducting planning for the first iteration, using their PI plans as a starting point.

Scrum of Scrums and Product Owner Sync (ART Sync)

After PI planning, the RTE typically facilitates weekly (or more frequently, as needed) SoS and Product Owner sync events. The SoS helps coordinate the dependencies of the ARTs and provides visibility into progress and impediments. The RTE, Scrum Masters, and others (where appropriate) meet to review their progress toward milestones and PI objectives, as well as dependencies among the teams. The event is timeboxed for 30–60 minutes and is followed by a ‘meet after’ where individuals who need to resolve specific problems or questions can remain behind. Figure 7-17 shows a suggested agenda for the SoS event.

Like the SoS, Product Owners and Product Management often hold a Product Owner sync. This event typically occurs weekly, or more frequently, as needed. The Product Owner sync is also timeboxed (30–60 minutes) and includes a meet after discussion. Sometimes the SoS and Product Owner sync are combined into one event, referred to as an ‘ART sync’ (Figure 7-17).

System Demo

A system demo is a significant event that occurs at the end of each iteration, which provides fast feedback about the effectiveness, usability, and releasability of the system (Figure 7-18). It offers an integrated view of the new features delivered by the ART over the past iteration, providing a fact-based measure of system-level progress and velocity within the PI.

This demo is done in a production-like environment (often staging) to receive feedback from stakeholders. It helps ensure that integration between teams on the same ART occurs regularly and that the emergent behavior of the full system can be evaluated. These stakeholders include the teams, Business Owners, executive sponsors, development management, and customers (or their proxies) who provide input on the fitness for purpose for the solutions being developed. The feedback is critical, as only they can give the guidance the ART needs to stay on course or make adjustments.

Prepare for the Next PI Planning Event

While we note this activity as a PI event, in reality, preparing for the upcoming PI is an ongoing process, with three primary focus areas.

• Alignment and organizational readiness for planning

• Backlog and content readiness

• Facility readiness—the actual logistics for the event

Since any one of these items can interfere with the potential outcome—a committed PI plan—careful consideration and planning is required for all three focus areas.

Inspect and Adapt

The I&A is a significant event held at the end of each PI, just prior to the next planning. It consists of three parts.

1. PI system demo. This demo is a little different from the regular system demos because it shows all the features that the ART has developed throughout the PI. During this demo, Business Owners collaborate with each team to agree on the actual business value achieved for their specific PI objectives.

2. Quantitative and qualitative measurement. Teams collectively review any quantitative and qualitative metrics they have decided to collect and then discuss the data and trends. One primary measure is the program predictability measure. The RTE summarizes the planned versus actual business value for each team’s PI objectives to create the overall program predictability measure.

3. Retrospective and problem-solving workshop. The ART runs a brief retrospective, the goal of which is to identify a few significant issues they would like to resolve. For addressing systemic problems, a structured, root-cause problem-solving workshop is then used to determine the actual root causes of a problem. The result is a set of improvement backlog items that go into the program backlog to be addressed during PI planning.

Continuous Exploration

CE fosters continuing research and alignment on what should be built. Design thinking continually explores market and customer needs and defines a vision, roadmap, and a set of features that meets those needs. During this period, new ideas are raised, refined, and prepared as a list of prioritized features in the program backlog. During CE planning, features are pulled into implementation, which begins the continuous integration process.

There are four main CE activities described in SAFe (Figure 7-21):

• Hypothesize describes the practices necessary to identify ideas and the measurements needed to validate them with customers.

• Collaborate and research describes the practices required to work with customers and stakeholders to refine the understanding of potential needs.

• Architect describes the practices necessary to envision a technical approach that enables quick implementation, delivery, and support of ongoing operations.

• Synthesize describes the practices that organize the ideas into a holistic vision, a roadmap, and a prioritized program backlog, and it supports final alignment during PI planning.

Continuous Integration

CI builds quality into the development process by continuously integrating the ongoing work of many Agile teams. All work is version controlled, and new functionality is developed and integrated into a full system or solution. It’s then validated in a suitable staging environment that ranges from cloud-based software systems to physical devices and device simulators.

SAFe describes four activities associated with continuous integration (Figure 7-22):

• Develop describes the practices necessary to implement stories and commit the code and components into the trunk.

• Build describes the activities needed to create deployable binaries and merge the development branches into the trunk.

• Test end-to-end describes the methods necessary to validate the solution.

• Stage describes the required practices to host and verify the system in a staging environment before production.

Continuous Deployment

CD captures the processes associated with moving solutions through staging into production environments. As with continuous integration, this varies substantially based on the kinds of solutions created and their solution context. Ensuring solutions are ready for customers requires deployment and monitoring to provide flexibility in controlling releases, rolling back a version, or installing incremental updates and patches.

CD consists of four key activities in SAFe (Figure 7-23):

• Deploy to production describes the practices necessary to deploy a solution to a production environment.

• Verify the system represents the practices needed to make sure the changes operate in production as intended before they are made available to customers.

• Monitor for problems describes the practices to monitor and report on any issues that may arise in production.

• Respond and recover describes the activities to rapidly remedy any problems that happened during deployment.

Release on Demand

As we described, RoD is the ability to make new functionality available to customers all at once or incrementally based on market and business needs. SAFe describes four RoD practices (Figure 7-24):

• Release describes how to deliver the solution to end users all at once or incrementally.

• Stabilize and operate describes the process needed to make sure the system is working well from a functional and nonfunctional perspective.

• Measure describes the practices necessary to quantify whether the newly released functionality provides the intended value.

• Learn describes how to decide what should be done with the information gathered and prepare for the next loop through the CDP.

RoD is critical to business agility, as the decisions of what to release to whom and when are vital business drivers. Release management provides governance for any upcoming scheduled or ad hoc releases. In a continuous delivery environment, participants closely monitor the release section of the program Kanban. This oversight ensures that items are released when needed to the right customers, that dark launches and canary releases are well managed, that hypotheses are evaluated, and that feature toggles are removed after production verification.

Program Epic Kanban System

ART initiatives that are simply too big to be completed in a single PI are called program epics. Also, some portfolio epics may need to be split into program epics to facilitate incremental implementation. Program epics may affect financial, human, and other resources that might be large enough to warrant a Lean business case, discussion, and financial approval from Lean Portfolio Management (LPM). Epics whose estimates exceed the established epic threshold will require review and approval.

Approving significant initiatives is one of the four critical Lean budget guardrails (see Chapter 9, Lean Portfolio Management, for more details).

Market Rhythm and Events Inform the Roadmap

To create the highest value for all stakeholders, customer-centric organizations leverage market rhythms and market events to inform the roadmap.¹² Simply put, the benefits of a solution can vary significantly based on when it’s released.

12. Luke Hohmann, Beyond Software Architecture: Creating and Sustaining Winning Solutions (Addison-Wesley, 2003).

• A market rhythm is a set of events that repeatedly occur on a predictable cadence. For example, retailers routinely prepare for the holiday shopping season by upgrading their systems to gain a competitive edge and to support significantly higher transaction volumes.

• A market event is a one-time future event, which has a high probability of materially affecting one or more solutions. They can be external, such as the launch of government regulations, or they can be internally created, such as a company’s annual user conference.