Measure and Grow

‘Measure and grow’ is the term we use to describe how a portfolio evaluates its progress toward business agility and determines improvement steps. It describes how to measure the current state and grow to improve overall business outcomes. There are two separate assessment mechanisms, which are designed for different audiences and different purposes.

1. The SAFe business agility assessment is designed for Lean Portfolio Management (LPM), Lean-Agile Center of Excellence (LACE), and other portfolio stakeholders to assess their overall progress toward the ultimate goal of true business agility.

2. The SAFe core competency assessments are used to help teams and trains improve on the technical and business practices they need to help the portfolio achieve its goals.

Each assessment follows a standard process pattern of running the assessment, analyzing the results, taking action, and celebrating the victories.

SAFe Business Agility Assessment

The SAFe business agility assessment is a high-level assessment that summarizes how Agile the business is at any point in time. The assessment report provides a visualization that shows progress measurements along the 21 dimensions. An example report is shown below in Figure 16-1.

Running the Business Agility Assessment

The business agility assessment contains a series of questions that help identify the current state of the portfolio. Typically, portfolio, business, and technology stakeholders take it, together with the LACE. After choosing the target audience and deciding which competencies the group should assess, it’s essential to set the context before starting the assessment, ensuring that all participants understand the 21 dimensions, as well as the SAFe terminology used.

Assessing business agility progress is not a trivial feat. Opinions abound, the data is lumpy, and the ways of working are evolving at the same time as the assessment is taking place. Therefore, simply sending the assessment out to various participants and asking them to fill in the data will probably not provide the right experience or accurate results. Instead, we recommend a facilitated session with someone trained in the nuances of SAFe and the assessment process. An experienced SAFe Program Consultant is probably a good choice.

Two assessment patterns can be used:

• Fill out the assessment individually and then come together to discuss and analyze the results

• Work together to discuss each question and reach a consensus on the score

Both patterns have their benefits and disadvantages. Either way, having a facilitator to set the context and answer questions during the assessment is vital. The facilitator also helps ensure that the assessment workshops are engaging and actionable.

Analyzing Business Agility Assessment Results

With the data from the assessment in hand, the next step is to analyze the results. During the analysis, it is important to identify significant variances in opinion. These might stem from a different understanding of the statement itself, or from disagreement about the current state in relation to a specific dimension. The goal is to explore the differences to get better alignment of where improvement is needed. This is a significant part of the collaborative learning experience.

Dimensions that the group has assessed as problematic can then be explored to understand the reasons that drove people to score themselves low. In addition to pointing out areas needing improvement, the assessment allows portfolios to see visible improvements in performance or ‘wins.’ The wins or small milestones, when multiplied, encourage teams to consolidate those gains and produce more change, as Kotter’s model suggests.

The facilitator should also be aware of the Dunning-Kruger effect,¹ in which people tend to assess their ability as greater than reality. In turn, dimensions that seem unnaturally high should be reviewed to ensure the group understands the meaning of the statements in question and are being realistic in their assessment.

1. https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

Taking Action on the Business Agility Assessment

Although high level, taking the business agility assessment is, in itself, a learning experience. Many questions directly set expectations of behaviors, activities, or outcomes that can be reasoned about and discussed. For example, a question about continuous learning such as “the organization provides dedicated time and space for people to explore and experiment” is fairly clear, and the implied corrective action is fairly obvious. Further in Figure 16-1, the enterprise scored low in LPM. That could be because they are ineffective at it, but it’s more probable that that enterprise hasn’t started that part of the journey yet. In most cases, a quick look at the implementation roadmap will identify some fairly obvious next steps, with the goal being to steadily improve proficiency across all seven core competencies.

The portfolio or the LACE should routinely re-evaluate their progress toward business agility, perhaps every other PI, and plan the next steps. The measurement frequency depends on the opportunities pursued and how fast the portfolio can reasonably achieve progress. Creating a baseline early on in the transformation, followed by periodic assessments, will illustrate improvement trends and allow everyone to communicate successes.

SAFe Core Competency Assessments

In most cases, assessing progress toward business agility spurs the enterprise to greater, and deeper, improvement efforts. That leads the business to explore and start to measure and take more specific action on some or all of the seven core competencies. Structured similarly to the business agility assessment, each core competency assessment has a set of statements, organized by dimension, that are rated on the same scale as the above. The questions go one step deeper to specific aspects and areas of opportunity and concern along each of the three dimensions of that specific competency. An example report is illustrated in Figure 16-2.

Running a Core Competency Assessment

As with the business agility assessment, the scope, audience, and process for an individual competency assessment must be purpose built. Perhaps low results in the Agile product delivery competency might suggest that each Agile Release Train assess its progress in that dimension. Or perhaps it’s an LPM assessment that needs to bring in the right stakeholders. In any case, all the guidance and caveats provided earlier still apply and attention to culture and careful facilitation is necessary to get the right experience and results.

Each of these more detailed assessments can be downloaded from the bottom of each competency article on the SAFe website.

Analyzing Results of a Core Competency Assessment

The results of a competency assessment are summarized along the three dimensions. But again, there is far more detail in the assessment and far more learnings than the figure alone implies. For example, here’s a sample of questions from the built-in quality dimension, which by themselves inform stakeholders and imply improvement activities:

• Teams share responsibility for design

• Teams reduce technical debt in each iteration

• Teams foster cross-training and T-shaped skills

• Continuous integration and automated tests run at team and system levels

To reiterate, taking the business agility or core competency assessment is not a routine, mechanical effort. It’s a facilitated collaborative session filled with learnings, and it sets expectations and communicates intent. Therefore, even the apparently simple act of taking the assessment will be a significant step in moving toward improvement.

Taking Action from a Core Competency Assessment

Based on the analysis of the core competency assessment, the team identifies the areas needing the most improvement. For example, in Figure 16-2, the built-in quality dimension of the team and technical agility competency has a low score, while the other areas appear solid.

Each competency article contains a ‘grows’ link, which opens a page that has recommendations for each dimension (Figure 16-3). The grows represent an activity (e.g., watch a video, run a workshop) that helps the team improve in that dimension.

The grows in Figure 16-3 are organized into three groups (core, intermediate, and advanced) to provide a logical order for selecting improvement actions. It’s also helpful to prioritize the opportunities using Weighted Shortest Job First (WSJF) and choose one or two activities to limit work in process and gain the most value in the shortest time. The top, highest priority items go into the appropriate backlog (e.g., LACE, program) based on the size and scope of the improvement.

Reinforce the Basics with Essential SAFe

The business agility and individual competency assessments provide a comprehensive framework to help the enterprise ‘measure and grow.’ But there’s often an intermediate path as well. In virtually every sport, when teams start to struggle, coaches will refocus their players on the fundamentals of their positions. This first step on the path back to winning is also often the same for organizations adopting SAFe.

At the beginning of the transformation, everyone was trained, and the initial emphasis was on learning the fundamentals of SAFe. However, as more trains are launched and the organization’s attention moves to new challenges, steps in the implementation roadmap may be skipped. Practices that were closely followed in the beginning may have been altered or discontinued. Whether it’s lack of understanding, a desire to shortcut the path to agility, or the natural ebb and flow of a company’s attention to performance, often one or more of the ten critical success factors in Essential SAFe (Figure 16-4) is not being followed.

Experience from thousands of SAFe implementations has shown that skipping any factor will lead to the portfolio achieving less than exemplary results. To overcome these challenges, the following ten critical success factors (Figure 16-4) are a subset of Essential SAFe that describe the basics necessary to achieve positive business outcomes.

Each of these elements is described next.

1. Lean-Agile Principles

SAFe practices are grounded in fundamental Lean-Agile principles. As organizations adopt SAFe, their continuous improvement activities find even better ways of working. These principles guide those efforts and ensure that adjustments are moving on a steady path to the ‘shortest sustainable lead time, with the best quality and value to people and society.’

2. Real Agile Teams and Trains

Real Agile teams and ARTs are fully cross-functional. They have everything, and everyone, necessary to produce working, tested increments of the solution. Self-organizing and self-managing, they flow value more quickly, with a minimum of overhead. Agile teams that cannot define, build, test, and deliver their work are not fully functional Agile teams. ARTs that cannot deliver solutions or parts of them are not fully capable ARTs.

3. Cadence and Synchronization

As described in Principle #7, cadence provides a rhythmic pattern—a steady heartbeat for the development process. It makes routine those things that can be routine. Synchronization allows multiple perspectives to be understood and resolved at the same time. For example, synchronization is used to pull the various assets of a system together to assess solution-level viability.

4. PI Planning

No event is more powerful in SAFe than PI planning. It provides the rhythm for the ART and connects strategy to execution by ensuring business and technology alignment. Aligning the entire ART to a common vision and goal creates energy and a shared sense of purpose.

5. Customer Centricity, DevOps, and Release on Demand

SAFe enterprises create a positive customer experience across their full set of products and services. They adopt a DevOps mindset, culture, and applicable technical practices to enable more frequent and higher-quality releases whenever the market demands. Customer centric, they apply design thinking to their work. These practices provide faster validation of hypotheses and produce greater profits, increased employee engagement, and more satisfied customers.

6. System Demo

The primary measure of the ART’s progress is the objective evidence from a working solution in the system demo. Every two weeks, the full system—the integrated work of all teams on the train for that iteration—is demoed to the train’s stakeholders. Stake-holders then provide the feedback the train needs to take corrective action and stay on course. This replaces other forms of governance that create additional work and impede flow.

7. Inspect and Adapt

Inspect and Adapt (I&A) is a significant event held every PI and is a predetermined time to reflect, collect data, and solve problems. The I&A event assembles teams and stakeholders to assess the solution and define the improvements and actions needed to increase the velocity, quality, and reliability of the next PI.

8. Innovation and Planning (IP) Iteration

The IP iteration serves multiple purposes. It acts as an estimating buffer for meeting PI objectives and provides dedicated time for innovation, continuing education, PI planning, and I&A. IP iteration activities support many Lean-Agile principles that enable business agility.

9. Architectural Runway

The architectural runway consists of the existing code, components, and technical infrastructure necessary to implement high-priority, near-term features, without excessive delay and redesign. Insufficient investment in the architectural runway slows the train and makes the ART’s delivery less predictable.

10. Lean-Agile Leadership

For SAFe to be effective, the enterprise’s leaders and managers take responsibility for Lean-Agile adoption and success. Executives and managers become Lean-Agile leaders trained in—and then become trainers of—these Leaner ways of thinking and operating. Without leadership taking responsibility for the implementation, the transformation will be unlikely to achieve the full benefits.

Anchor New Behaviors in the Culture

Implementing SAFe and achieving mastery of the seven competencies will inevitably shift the culture of the organization. Anchoring this shift to make it permanent is critical to ensuring that the organization keeps progressing and doesn’t regress to old patterns of behavior. As portfolios advance toward higher degrees of mastery, there will be a natural tendency to assume that the new way of working has become habit and to shift the focus of the organization to the ‘next big thing.’

But beware. Until all of the new principles and practices spread throughout the entire portfolio become the default way of working, there will be an ongoing risk that some or all of the organization’s gains will be lost. The portfolio could easily revert to legacy mindsets and practices. Changes in leadership, threats that create an emergency response environment, or the organization not having enough time and experience for the new habits to take root are all possible causes.

So, how can this pitfall be avoided?

Once again, the wisdom of W. Edwards Deming points the way. “Transformation is not automatic. It must be learned; it must be led.” Leaders must do more than ‘change the system.’ Leaders have to understand the principles and practices of change leadership and organizational change management. They must become curators, caretakers, and defenders of the new way of working. When the heat is on and the pressure to revert to old habits escalates, everyone will look to leaders from the top of the organization down to see how they respond. Has leadership transformed how it thinks, acts, and makes decisions—even in moments of crisis? When leaders demonstrate that real change has occurred and going backward is not an option, no matter the circumstances, the changes become part of the organization’s DNA. Then, the new way of working is likely to withstand similar challenges in the future.

Many of these new behaviors are modeled and exemplified by two specialty roles: the Scrum Master and the RTE. It takes a combination of coaching, experience, and training to develop them.

Apply Learnings Across the Enterprise

It’s common for the world’s largest organizations to have many portfolios. However, success in one portfolio does not ensure success in the others. As the initial portfolio adopting SAFe progresses toward business agility, the next step of accelerate is to leverage the insights and successes of the pioneering organization to transform the remaining portfolios. The recommended pattern is to provide change agents from the initial portfolio with the opportunity to help transform future portfolios, bringing with them all of the experience and understanding of implementing SAFe. For this pattern to succeed, organizations need to invest in cultivating the next generation of leaders in every role so they are prepared to step in when their counterparts move on to launch the transformation in other portfolios. Planned well, these transitions can be smooth and can create great opportunity and upward mobility for these leaders. They can also assure that the existing portfolio continues making progress toward the never-ending journey of mastering business agility.

Summary

Although it might appear so in the implementation roadmap, the accelerate step of SAFe is not the last in the transformation journey. Rather, it’s the beginning of a new and larger journey of relentless improvement.

The emerging Lean enterprise has started to build a new operating model. Persistent progress is becoming the norm, but it can’t be taken for granted. Accelerating the transformation requires continuous retrospection and identification of opportunities for improvement. The business agility and core competency assessments provide a means to measure this progress and identify areas for growth. In addition to the ten critical ART success factors, specific ‘grows’ provide activities to increase proficiency in each core competency. Over time, the new behaviors that emerge will become anchored in a new organizational culture, thereby creating the opportunity to spread learnings wider across the enterprise.

Taken together, the ‘accelerate step’ and the ‘measure and grow’ guidance provide a comprehensive strategy that will help the enterprise achieve the full business benefits of a SAFe implementation.