Continually Refine Fixed and Variable Solution Intent

The systems engineering discipline has traditionally applied the familiar ‘V’ life-cycle model.² This model describes the critical activities for building large systems from inception through retirement. Flow-based systems like the Scaled Agile Framework (SAFe) describe these same activities, but they occur in smaller batches and continuously throughout the life cycle, as shown in Figure 8-2.

2. https://www.sebokwiki.org/wiki/System_Life_Cycle_Process_Models:_Vee

SAFe’s flow-based model (Figure 8-2 right) enables engineers to continually perform the following types of activities simultaneously for each increment: explore innovative ideas, refine features, integrate and deploy features, and release value on demand.

Solution Intent

Due to the complexity of the systems being built, solution intent (Figure 8-3), a central knowledge repository, is used to store, manage, and communicate what is being built and how it will be built. Solution intent provides many benefits to the systems engineering process.

• Maintains a single source of truth regarding the intended and actual solution behavior

• Records and communicates requirements, designs, and system architecture decisions

• Facilitates further continuous exploration and analysis activities

• Aligns the customer, Agile teams, and suppliers to a common mission and purpose

• Supports compliance and contractual obligations

Developers of complex systems must constantly know two things: what the system currently does now and what changes are intended for the future. Knowledge of both the current and future states can be captured in any form suitable and includes three primary elements: specifications, designs, and tests. As the future intent is implemented and evolved, it then gets recorded as the current system state.

Traceability helps confirm that life- and mission-critical systems (and others governed by regulation) are built in exact agreement with intended behavior. It connects all the solution intent elements and the system components that realize its full behavior. The solution intent is created collaboratively and evolves based on learning.

Solution Context

Every solution operates within the context of its environment (e.g., cloud, factory, home, on-premise servers, system of systems). Since a system is often installed and maintained in a different environment than it was developed, the solution context is needed to understand the requirements, usage, installation, operation, and support of a solution. As a result, understanding the solution context is critical to reducing risk and achieving fitness for purpose. Aligning the solution intent with the solution context (Figure 8-4) requires a customer-centric mindset and collaboration.

For example, customers participate in PI planning and solution demo events as frequently as possible to assure alignment. As solutions increase in size and complexity, the customer frequently integrates them into their specific solution context. Ideally, these integration cycles are aligned with the cadence of the train, so solution increments can be built, integrated, and deployed based on correct assumptions, allowing frequent validation of the solution under development.

Create Minimal but Sufficient Documentation

Document-based approaches to managing solution intent and context do not scale. Indeed, they quickly become obsolete and inconsistent with one another. The alternative is a set of related digital models that define, design, analyze, and document the system under development. Some models specify system requirements and design, while others are domain-specific (e.g., electrical, mechanical, or some systems property).

Connecting models ensures system completeness and coverage (Figure 8-5). For example, engineers can see whether all requirements are implemented by components and covered by tests. They can identify the impact of replacing a component, including what requirements and tests are affected. Together, these models record and communicate what the system does and how it does it to a diverse set of key stakeholders.

By using a more flexible approach to managing and communicating the specifications, solution intent aligns everyone to a shared direction. Its companion, solution context, defines the system’s constraints—deployment, environmental, and operational.

Because solution intent and context for large solutions flow down into components and subsystems, they offer teams the implementation flexibility needed to make local requirements and design decisions. For example, Figure 8-6 shows that as downstream teams (system, subsystem, and component) implement decisions, the knowledge gained from continuous exploration, integration, and deployment returns feedback upstream and moves decisions from variable (undecided) to fixed (decided).

Apply Multiple Planning Horizons

Building massive, technology-based, innovative systems has high uncertainty and risk. The traditional way to reduce risk has been to develop detailed, long-range plans up front. In practice, however, gaps in specifications, evolving business needs, and technical issues can quickly make these plans obsolete. Instead, Agile teams and trains use backlogs and roadmaps to offer a more flexible approach for managing and forecasting work, enabling teams to deliver the most value each increment.

Effective road mapping efforts require an understanding of the appropriate time horizon. If the horizon is too short, the enterprise may jeopardize alignment and the ability to communicate future features and capabilities. Too long, and the enterprise is basing assumptions and commitments on an uncertain future. Multiple planning horizons provide a balance (Figure 8-7) between short- and longer-term planning. The outer levels of the planning horizon are longer-term and describe behavior that is less defined and less committed, while the inner levels are nearer term, defining better understood and more committed solution behavior (Figure 8-7).

Architect for Scale, Modularity, Releasability, and Serviceability

Architectural choices determine the effort and cost required for future changes and are critical economic decisions. Software development can leverage frameworks and infrastructure that provides proven architecture components out of the box. Conversely, builders of large, cyber-physical systems need to define their own components, applying intentional architecture and emergent design. This encourages collaboration between architects and teams to create a resilient system that enables teams and trains to independently build, test, deploy—and even release—parts of large solutions.

Developers and engineers apply set-based design (Principle #3) to keep requirements and design options flexible for as long as possible during the development process. Instead of choosing a single point solution up front, they identify and simultaneously explore multiple options, eliminating poorer choices over time. This enhances flexibility by committing to technical solutions only after validating assumptions, which produces better economic results.

With the right architecture, elements of the system may be released independently. Figure 8-8 illustrates an autonomous delivery system that was architected to enable system elements to be released independently. Here are some examples:

• A software application running in the cloud deploys continuously and releases frequently.

• Embedded code running in a vehicle is updated over the air and deployed and released less frequently.

• Hardware updates (e.g., sensors, CPUs) often require taking vehicles out of service and possible recertification, so they are updated less frequently.

Although design choices that enable continuous value delivery may lead to higher unit costs for more adaptable hardware and vehicle communications, they can significantly lower the total cost of ownership and the product’s lifetime value as they support more frequent updates and enhance future flexibility.

Continually Address Compliance Concerns

The failure of a large system can have unacceptable social and economic consequences. To protect the public safety, these systems are typically subject to significant regulatory oversight and compliance requirements. Quality Management Systems (QMS) help system builders ensure quality, reduce risk, and define practices that confirm safety and appropriate solution behavior.

However, most quality management systems were created based on traditional, sequential development approaches that often assumed (or mandated) up-front commitment to specifications and design, detailed work breakdown structures, and document-centric, phase-gate milestones (top, Figure 8-9).

In its place, a Lean QMS (bottom, Figure 8-9) is implemented to help the system builder meet compliance goals and make progress more visible. These practices include the following:

• Build the solution and compliance incrementally

• Organize for value and compliance

• Build quality and compliance in the daily work

• Continuously verify and validate the work

• Release continuously validated solutions on demand

These practices assure that compliance occurs throughout the product development process, instead of only during inspection at the end, which often causes significant rework and delays.

Build and Integrate Components and Capabilities with Solution Trains

Solution Trains coordinate multiple ARTs and suppliers to build complex solutions that require hundreds or even thousands of people to build (Figure 8-10).

As systems scale, alignment becomes more critical. To plan, demo, learn, and improve together, Solution Trains align all of their ARTs to a common cadence. They integrate their solutions at least every PI to validate that they are building the right thing and verify technical assumptions.

More frequent integration is possible for teams building software solutions (see the “Apply Continuish Integration” section below). ARTs that have longer lead times for components (e.g., hardware) deliver and demo incrementally through proxies (e.g., mockups, stubs, prototypes), which can integrate with the overall solution and support early validation and learning.

Solution Train Planning

Solution Train planning provides a way for ARTs and suppliers to collaboratively build a unified plan for the next PI. It also fosters team-building across the Solution Train, which helps create the social network necessary to achieve high performance. Solution Trains introduce two additional events (pre- and post-PI planning) to coordinate their ART’s PI planning as shown in Figure 8-11.

The pre-PI planning event brings together stakeholders from all parts of the Solution Train to create a clear vision and context for the upcoming increments. Inputs include the current solution roadmap, vision, solution intent, and the top capabilities from the solution backlog. Attendees include the following:

• Solution Train leaders. This includes the Solution Train Engineer (STE), Solution Management, Solution Architects and Engineers, solution system team, and possibly customer representatives, particularly for bespoke systems.

• Representatives from ARTs, suppliers. Typically, this includes RTEs, Product Management, System Architects and Engineers, and other key stakeholders.

The post-PI planning event summarizes the results of each ART’s PI plans to ensure alignment for the upcoming increment. It creates agreement on the solution PI objectives to be implemented and presented at the next solution demo.

Pre- and post-PI planning occur as close to the ART planning events as reasonable. While not always feasible, it is desirable to have all the ARTs plan at the same time. This allows for a joint solution-level context and vision briefing. And it enables a solution-level management review for any adjustments before the second day of planning as shown in Figure 8-12.

The practical logistics of Solution Train planning may limit solution stakeholders from participating in each ART’s planning events. However, it’s critical that key stakeholders participate in as many of the ART PI planning events as possible—particularly Solution Management, the STE, and Solution Architecture and Engineering. Typically, these solution stakeholders may attend by circulating among the different ART PI planning sessions. Suppliers and customers play a critical role here as well, and they should be represented.

The Solution Train planning events deliver similar benefits as PI planning for a single ART. Further, the post-PI planning event assures demand matches capacity and removes potential excess Work In Process (WIP). A successful event delivers three essential artifacts:

• A set of ‘SMART’ Solution Train PI objectives, which includes any planned but uncommitted goals, and the business value set by Business Owners

• A solution planning board (Figure 8-13), which highlights the capabilities and their dependencies, anticipated delivery dates, and any other relevant milestones

• A commitment based on the confidence vote for meeting the Solution Train PI objectives

Apply ‘Continuish Integration’

In the software domain, continuous integration is the heartbeat of continuous delivery. It verifies changes and validates assumptions across the entire solution. Development teams invest in automation and infrastructure that builds, integrates, and tests every developer change, providing near-immediate feedback on errors.

Large, cyber-physical systems are far more difficult to integrate continuously: long lead-time items may not be available, integration spans organizational boundaries, and end-to-end automation is difficult to accomplish.

Instead, ‘continuish integration’ addresses the economic tradeoffs of frequent integration versus delayed knowledge and feedback. The goal is constant, partial integration with at least one full solution integration during each PI (Figure 8-14).

When full integration is impractical, partial integration significantly reduces risks. Agile teams and trains integrate and test in a smaller context and rely on the system team for more extensive end-to-end tests with authentic production environments. This allows testing a partial scenario or use case, or testing with virtual and emulated environments, test doubles,³ mocks, and other prototypes. These practices reduce the testing time and costs for teams and trains.

3. “A test double is a stand-in for a real object in a test, similar to how a stunt double stands in for an actor in a movie” (source: https://testing.googleblog.com/2013/07/testing-on-toilet-know-your-test-doubles.html).

Manage the Supply Chain with Systems of Systems Thinking

Building large and complex systems requires integrating solutions across a substantial supply chain. Clearly, supplier alignment and coordination are critical to solution delivery. As a result, strategic suppliers participate in SAFe events (PI planning, system demos, I&A), use backlogs and roadmaps, and adapt to change. Agile contracts⁴ encourage this behavior. The supplier’s Product Manager and System Architect or Engineer continuously align the backlog, roadmap, and architectural runway with those of the overall solutions. Similarly, customer and supplier system teams have to share builds and tests to ensure that integration handoffs are smooth and free of delays.

4. For more information on Agile contracts, see https://www.scaledagileframework.com/agile-contracts/.

Figure 8-15 shows a complex integration example for a large supply chain. The Product Manager of the Flight Controls component needs to continually align the backlog and roadmap to balance the needs of multiple customers (e.g., Boeing, Airbus, and Bombardier). Any product changes can ripple across the roadmaps of the extended supply chain, as this example illustrates.

To support multiple customers or customer segments, suppliers may offer product variants.⁵ The decision to create a new variant is significant since each must be individually tested, released, maintained, and supported. Due to the complexity, it’s easy to create a new customer variant without having a strategy for coordination and understanding the entire life-cycle costs. There are two common patterns for coordinating customers and suppliers (Figure 8-16).

• Clone-and-own model. This is a common reuse practice that creates a product variant for each new customer based on assets from existing products. New variants are copied and adapted from existing products. This approach limits economies of scale since enhancements and defect fixes in one variant do not automatically propagate to the others and must be manually repeated. Over the long term, this often results in overall higher costs and lower quality.

• Product line (or platform) model. In this model, all developers work on a common architecture that creates a group of related product variants, which can be adapted for individual customers or customer segments. This approach necessitates a shared vision and roadmap that satisfies the needs of multiple customers. The supplier continually aligns backlogs with their customers and delivers new versions incrementally. While this approach may struggle to balance the needs of all customers, it provides opportunities for quality and technology investment that typically do not exist in clone-and-own.

5. A product variant is a version of a product that has different features or that operates in a different solution context.

‘Internal open source’ is a growing supplier collaboration practice where changes made to a customer variant are pulled back into the main product. This practice reduces delays to make changes while balancing support for multiple customers. The supplier is responsible for overall product quality and ensuring appropriate governance practices are followed, including who can make and contribute changes.

Build a Continuous Delivery Pipeline

Traditional large system development focuses on building the system right the first time and minimizing changes once the system is operational. After all, innovations and enhancements require a significant system upgrade effort. However, today’s system must always evolve.

Figure 8-17 shows a typical CDP where every small developer change automatically runs a build process to create packages that are deployable and tested, giving feedback to developers within a few minutes. Packages that pass these tests then undergo more comprehensive automated acceptance testing. Once these packages pass all the automated tests, they become available for self-service deployment to other environments that support exploratory testing, usability testing, and ultimately release.

Many technologies enable the pipeline. Although the software technologies are well-known and becoming standard, the cyber-physical community is just beginning to implement emerging hardware technologies that assist the CDP.

Significant cyber-physical systems must also use CDPs to support continuous releases of new functionality. This requires investing in automation and infrastructure that can build, test, integrate, and validate small developer changes on an end-to-end staging environment or a close proxy. It also requires an architecture that can leverage technologies such as over-the-air updates and programmable hardware to enable faster deployment and release in the operational environment.

Development of the system and the CDP should start and evolve together. The following additional practices support the creation and use of the CDP:

• System engineering activities for analysis and design are performed in small batches to flow through the pipeline quickly.

• Planning includes building the pipeline as well as the system.

• ‘Continuish’ integration creates the automation and environments that can flow changes through the pipeline.

Evolve Deployed Systems

Investment in the CDP changes the economics for going live. A fast, cost-effective pipeline means a minimum viable system can often be released early and evolve. This delivers feedback and insights much earlier with less investment, possibly generating revenue sooner. Updating live systems is not new. For example, satellites have been launched before the software was fully ready. For businesses, the goal is to deploy the solution, quickly gain feedback and insights, deliver value, and reach the market before the competition.

Systems should be architected to support continuous deployment and release on demand. To create components more quickly and economically, hardware modeling languages, additive manufacturing, and robotic assembly can enable ‘hardware as code.’ Certain design decisions simplify system evolution, such as programmable versus application-specific integrated circuits, non-permanent fasteners, and allocation of system functions to upgradeable components.

Engineers are also exploring ways to adopt well-known software DevOps practices. For example, the blue-green deployment technique reduces downtime and risk by running two identical production environments—one for staging and one for live operations. This approach is being used on large systems like Navy ships, where the cost of redundant hardware is offset by releasing new capabilities into the operational system years earlier.