6
Building the Agile Release Train
Launching an Agile Release Train (ART) can be a daunting task. There are numerous things to consider when building your ART, including the following:
- Its structure and composition
- Why it’s important to align your ARTs to Value Streams
- Key roles in the ART
- What a Troika is, who is on the Troika, why the Troika is important, and the influence needed for success
- What a System Team is
We will take a look at each of these areas and help you understand some key considerations, regardless of whether you are launching your first ART, recovering an ART, or maturing an ART.
Why Your Train isn’t Your Department
One of the first mistakes many organizations make when standing up an ART is they align it with their existing organizational structure. After all, our organizations have evolved into a system that works. We have an organizational structure that we continually refine (reorganize) to make sure we are optimally aligned. However, have you stopped to consider the reason we seem to be in a constant state of re-org?
How should we Build our ART?
Let’s start by defining what an ART is. An ART is a virtual organization of 5 to 12 long-lived Agile Teams that consist of 50 to 125 people working to deliver a common solution on cadence. Successful ARTs are aligned with a shared mission or vision.
Now that we know what an ART is, we need to determine whether we need one.
A common mistake that many organizations make is assuming that they “need” an ART, and they stand one up for every project they undertake. The organization identifies the project and then finds the “resources” to work on that project.
With SAFe®, we take a different approach and create our teams and ARTs for our Development Value Streams; see Chapter 12 for additional information. This naturally allows us to align the ART so that it works with a specific domain and take work to the ART rather than standing up and tearing down ARTs as projects start and stop.
Cross-Functional
When creating our ART, the intent is that our ART can support itself. This means we want our ART to be organizationally cross-functional. We will need people from all parts of the organization, including Business, Engineering, Testing, Operations, Security, Compliance, and so on. We often encounter situations where these parts of the organization typically don’t work together daily and have their own hierarchy and even culture.
As we launch our ART, we likely won’t have the ART composition correct on our first attempt and that is OK – it will evolve. The intent is to minimize hand-offs and subsequent delays. Our non-siloed cross-functional teams will drive this in conjunction with a strong Continuous Integration/Continuous Delivery (CI/CD) pipeline with DevOps support. Chapter 7 has additional information on CI/CD and the Continuous Delivery Pipeline (CDP).
Note that even cross-functional ARTs will still need to interface with the rest of the organization and will need a System Team and will likely have dependencies on Shared Services Teams.
Dedicated and Long-Standing
Our ART should have dedicated team members. This means that the individual is allocated to the ART and their team 100%. We want to minimize task switching and keep teams together for as long as is reasonably possible.
Pro tip
If I had to pick one battle to win when setting up an ART, it would be having dedicated team members. Too many times I see organizations allocate a person 25% to the ART. 25% of the time, they attend the events and are left with an hour or two of actual time to work on stories. How much of a story can you realistically get done in 1 or 2 hours?
Communicative
Our ART needs to be able to communicate freely and effectively across organizational boundaries. People from Business need to be able to regularly talk with people from IT and subsequently people in Operations and people in Marketing and throughout the organization. This communication must flow organically without you having to work up and down the organization chart.
Conway’s Law
Conway’s Law [1] states that an organization’s output is directly related to how it communicates internally. What we have discovered is that these communication hierarchies are often significant barriers to flow. Organizations with successful ARTs have figured out the nature of a Dual Operating System and understand that our ARTs should be built around our social structures (how we talk and chat and work with each other informally) as opposed to our organizational hierarchies.
Maturity
ARTs mature and evolve. ARTs, like teams, follow Tuckman’s model of Forming, Storming, Norming, and Performing, as explained for teams in Chapter 2. As a Coach and/or Release Train Engineer (RTE), you want your ART to become high performing. One thing to keep in mind is that the ART will mature more slowly than a team will due to its size.
Pro tip
I liken the maturity of an ART to that of a child; each PI is like a year of child development. When that ART is in its first PI, I liken it to a newborn baby. It needs constant care, feeding, and changing.
The second PI is like a 1-year-old; maybe a little less care, but it still needs to be fed. It’s maybe just starting to crawl.
The third PI is like having a 2-year-old with constant “Why” questions.
By the time we get to the fifth PI, the ART is like a kindergartener. It can get dressed on its own, but its clothes are likely mismatched and its shoes are on the wrong feet.
It’s about 2 years in, around PI 9 or 10, that the ART can largely take care of the daily activities with help and support for the big items.
It takes time for ARTs to grow and mature. The longer you keep the ART stable, the better it will execute and deliver.
When our ARTs aren’t communicative, cross-functional, dedicated, long-standing, and mature, they are inefficient. The ART will be plagued by corporate politics, it will grow and shrink with each budget cycle, and the churn and overhead of standing up and down ARTs in the same fashion as projects are just wasteful.
How Identifying the Correct Value Stream Impacts the ART
Pause for a moment and ask yourself, what is the product or the solution that my company delivers? Now, consider for a moment, are the teams and ARTs that you’re working with directly supporting that product? Is everybody aligned to ensuring that the product is successfully delivered every day? Oftentimes, you will find that a company’s core competence is X, which isn’t necessarily what the teams and trains are attempting to build.
Let’s consider an Airline company. Its product is flying passengers from Point A to Point B. It also has a mobile app and a website that lets users book tickets, change reservations, check in, and so on. Like most companies, it is hierarchically organized to support flying passengers. However, when it embraced the concept of the Dual Operating System and socially aligned its ARTs with the products it was creating, such as a mobile app versus a plane maintenance scheduling application, they were inherently aligned with their Value Streams.
Pro tip
As a Coach, I recommend leveraging the SAFe® Value Stream Identification Workshop to identify your Value Streams and ARTS. This is an invaluable resource available to SPCs in good standing.
How should we Identify our Value Streams and what happens if we don’t?
Simply put, we should align our ARTS with the products or solutions that we are attempting to deliver, which, in turn, typically align with our Development Value Streams.
SAFe® recommends we start by identifying our Operational Value Stream and then determine which systems support it. Next, we identify the Development Value Streams that support those systems and subsequently identify the ART(s) for them:
Figure 6.1 – Example Value Streams and ARTs identified in the Value Stream Identification Workshop (© Scaled Agile, Inc.)
When we don’t align closely with our Development Value Streams, then our ability to deliver work continuously and on cadence is severely hampered. The teams will often be hampered by dependencies and/or will duplicate architectural efforts and have limited learning.
The smaller the organization, the less this will impact you as your scale is smaller. If I only have one ART or even two ARTs, then the impact of misalignment is significantly smaller than if there were 15 ARTs.
Pro tip
If you can’t re-align your ARTs, strive to reduce and minimize the delays in your system. Leverage Value Stream Mapping to identify your biggest bottlenecks and start there.
Our primary intention is to effectively flow work through our system. It’s like water flowing through a pipe. If we have a straight, clear pipe with the proper angle, water flows easily. Every bend, change in elevation, or even build-up slows down the flow. We want to work to eliminate the blockages, bends, and improper elevation.
Ultimately, we won’t get the benefits of Agile that we are looking for without proper alignment. If we don’t align our ARTs with the Development Value Streams, we will continually be challenged with dependencies, lack of overall alignment, duplication of effort, and significant challenges in the portfolio area, particularly when it comes to prioritizing the Portfolio Backlog and portfolio funding and budgeting (see Chapter 13).
Pro tip
Value Stream Identification and Value Stream Mapping are two separate activities. Value Stream Identification is used to identify what your Value Streams are and then identify what teams and ARTs are necessary to support the identified Development Value Streams. See the Value Stream Identification Workshop available in the SAFe® toolkits.
Value Stream Mapping involves looking at the steps in your Development Value Stream. SAFe® covers this activity in the DevOps class and has a Value Stream Mapping Workshop Toolkit that is available.
Release Train Engineers, Product Management, and the System Architect – Senior Leaders
An empowered high-functioning team consisting of your Release Train Engineer (RTE), Product Management, and System Architect is critical to the success of the Agile Release Train. We often refer to this group as the Troika; our group of three responsible for the success of the ART.
The Troika
Our Troika comprises the Release Train Engineer, Product Management, and System Architect who collectively lead and guide the ART. Together, this group ensures the success of the ART by making sure the teams are developing the right things, at the right time, at the right level of quality, and with the right alignment.
In a successful ART, the influence of the Troika is critical. Collectively, they influence the entire organization.
We look to Product Management to influence the following:
- Leaders to undertake an Epic
- Teams to deliver the product
- Product Owners to execute the vision
- Architecture to push technology forward
We look to the System Architect to influence these aspects:
- Alignment across ARTs and the potential organization
- Collaborate with Release Management
- Participate in System Demos explaining the value of Enablers and Non-Functional Requirements (NFRs)
- Participate/lead an Architecture Sync
- Drive the System Team and ART teams in creating an Architectural Runway
- Alignment with the Enterprise Architectural Vision
We look to the RTE to influence the following:
- Alignment across the teams and organization
- Participation of leaders in key events
- Continuous collaboration, growth, and maturity of the teams
- Ensuring the teams and ART have what they need to be successful
- Removing blockers and impediments
If the Troika members lack sufficient influence and even authority, you will quickly find that the progress of the ART will grind to a halt.
The decisions that this group makes likely have long-term impacts on the organization. They need to be involved not only in the day-to-day work of the teams but be part of the decisions that are being made strategically about the business in general.
Pro tip
Avoid the mistake that many organizations make in assuming that these roles are staff-level roles. Successful organizations recognize their value and often, these roles are filled by Director-level roles and above.
A defined and sufficiently powerful Troika is critical to an ART’s ability to deliver high-quality, committed value. Each member of the Troika has an important role in ensuring the success of the ART and it takes work and effort from all three roles for success.
Pro tip
As a Coach it is important that you ensure the success of the individuals in these key roles. You may want to help identify and recommend individuals for each role.
Let’s take a deeper dive into each of the roles on the Troika, starting with the RTE.
The Release Train Engineer (RTE)
The RTE is like the quarterback of your football team. He or she is calling the plays, getting the players aligned, and making sure that the ART is scoring goals by delivering value.
There is a perception that the RTE is just the Scrum Master/Team Coach for the ART. While this is technically true, the RTE role goes beyond the expectations of a typical Scrum Master/Team Coach. The RTE is the ultimate Servant Leader. A successful RTE often has prior management experience and domain knowledge.
When identifying an RTE, look for someone who can do/has the following:
- Create an environment of respect and mutual influence
- Understand and empathize with others
- Strong listening skills
- Excels in problem identification and decision-making
- Coaches and leads
- Appreciates others and leverages support
- Is very organized and diligent with follow-up
- Thrives in challenging situations and with constant change
- Has the ability to communicate at all organizational levels and audience as appropriate
- Can anticipate what is coming down the tracks
The RTE often interfaces not only with the ART but also with Stakeholders, customers, and the executive leadership in the organization. The demeanor, level-headedness, and eloquence of the RTE are critical.
There are many different career paths successful RTEs follow, but most individuals come with backgrounds in management and were former Program or Project Managers, Scrum Masters/Team Coaches, or Lean-Agile Coaches.
The RTE needs to be able to command a room and not be intimidated by large groups of people. The RTE needs to be witty and entertaining and the life of the party, particularly for PI Planning.
Story from the real world
I once worked with an RTE who was new to the role. She was an amazing Scrum Master and when the opportunity arose for her to become an RTE, she was the natural selection. She was knocking it out of the park when it came to handling the day-to-day aspects of the ART, but when the first PI event occurred, where she was in front of 150 people and leading the show, she froze. She looked down at her feet and notes, barely made eye contact with the audience, and spoke in an extremely monotone voice. One of her fellow Scrum Masters took pity and helped facilitate the remainder of the PI event.
I began working with her shortly thereafter and we discussed her fears and challenges with being in front of a large group of people. We worked on this over the next several months and one of the activities we tried was reading children’s books out loud with lots of voice inflection.
We practiced reading to pets, nieces and nephews, co-workers’ children, and even an assembly at a school. By the time we got to the next PI event, the RTE had discovered that if she could read a book to a group of children, she would be able to lead the next PI event – and she did so with flying colors.
While this approach may seem a little unorthodox, we did what worked for her. I have since used this approach with others, Product Owners in particular, who are presenting often at System Demos and lack voice inflection and therefore lose their audience and often insight and feedback from their customers.
By practicing reading story books with character voices, they start to naturally begin to add more inflection in their voice as they tell the story of what work the team delivered and the functionality of the system.
Try it out yourself – you may find it helps you as well!
On occasion, a former Program or Project Manager can make an excellent RTE; however, a shift in mindset away from command and control and strict schedule adherence must occur. We need to reinforce that RTEs need to fundamentally shift their behaviors and work patterns when coming from those types of roles.
SAFe® outlines the RTE needs to move away FROM and TO the following behaviors:
- From coordinating team activities and contributions to Coaching the teams to collaborate
- From deadlines to objectives
- From driving toward specific outcomes to being invested in the ART’s overall performance
- From knowing the answer to asking the teams for the answer
- From directing to letting the teams self-organize and hit their stride
- From fixing problems to helping others fix them
Responsibilities of a Release Train Engineer
The responsibilities of the RTE can’t be underrated. Ensuring that everything and everyone is aligned and on track is a full-time job. It’s a version of extreme cat herding.
The RTE needs to be able to also understand and navigate through various Coaching stances to determine when to engage versus when to let teams and individuals struggle a bit to achieve learning.
There are five primary activities that the RTE is focused on:
Figure 6.2 – The RTE’s primary responsibilities (© Scaled Agile, Inc.)
The RTE also needs to ensure they don’t get bogged down by practices. Ensuring a solid understanding of the principles and staying in alignment is key. Overly rigid practices and not having an awareness of when and how much to flex known successful practices are key for an RTE.
There is a common misperception about RTEs that they are responsible for the release of the system/solution being developed. While the RTE has a hand in the release, there is no expectation they manage and coordinate it or that they are the single accountable person that ties up all the loose ends. They rely heavily on the System Team, Product Management, the System Architect, and the teams to release systems. The RTE will help coordinate the activities and ensure that impediments and blockers are removed or escalated appropriately, but the RTE should not be the person figuratively pushing the button to release.
Product Management
Product Management is a person or group of people responsible for knowing what needs to be built and ensuring that the ART can deliver the solution in incremental units of value. Product Management owns the product vision, and to do that effectively, it needs regular customer exposure to support it.
In many ARTs, Product Management is a single person. However, in highly complex and/or distributed environments, it takes several individuals, often with specialized knowledge, to define the product. In this scenario, we recommend that one person has the overall final decision and lead responsibility.
Product Management needs to be quite personable and easy to work with. When identifying Product Management, look for someone who has the following qualities:
- Has good market and domain knowledge
- Is trusted by peers and colleagues
- Has experience with messaging complex information
- Understands how to position the solution within the market
- Has proven success in working toward consensus
- Is forward thinking and curious about customer needs
- Balances competing priorities
- Is able to admit when a solution doesn’t meet the hypothesis and pivot appropriately
The demeanor, level-headedness, and eloquence of Product Management are critical as they interface and collaborate across the organization and with customers and clients.
Product Management must collaborate with many roles. They have lots of conversations with all levels of the organization and frequently meet with customers and get insight and feedback on the products they are developing:
Figure 6.3 – Product Management key collaborations (© Scaled Agile, Inc.)
There are many different career paths successful Product Managers follow, but most individuals come with backgrounds in management and product development. Some were Product Managers, Domain Experts, Marketing Analysts, Subject Matter Experts, or Product Owners.
Due to these frequent interactions, Product Management often become overwhelmed by the responsibilities of the day-to-day working of the ART. This is where the Product Owners and Product Management relationship is important.
The Responsibilities of Product Management
SAFe® outlines five main responsibilities for Product Management:
Figure 6.4 – Product Management responsibility areas (© Scaled Agile, Inc.)
The Product Management role is particularly challenging and finding the right person or people is often one of the biggest challenges. Product Management needs to be able to balance the long-term vision of the product with the short-term ART deliverables. The right person will possess the necessary soft skills, product knowledge, and leadership qualities to drive the product forward and deliver value to the customers and the organization.
Story from the real world
I worked with a particular client and re-launched an ART. The ART was about two weeks from their third PI Planning. The Product Manager, while an expert in his field, had not sufficiently prepared a backlog and was struggling to help the Product Owners identify the work for the next PI. The Product Manager knew what he wanted for the final product but struggled to break it into pieces that the teams could consume and deliver incrementally.
I met with the Product Manager and quickly learned that he was attempting to define and build the product step by step. For example, if we were building Amazon’s website, we would completely build the home page, then all the individual product pages, the user profile, subscription services, and music, all before building the cart to check out.
We worked quickly with the Product Owners to identify some critical spike areas for the teams in the next PI. We were able to look at the system as a whole and identified smaller parts from each functional area that would allow us to see work flow through the entire system.
During the PI, the Product Manager and I worked diligently with the Product Owners and System Architects to develop a path through the system that would cover all the key components. We called this our through-line; however, some folks might consider this the happy path. The Product Manager was then able to add different pieces of functionality and complexity iteration after iteration until the system was good enough for release.
The Product Manager shared that by breaking the work down and identifying the simplest path first, as opposed to building out everything completely step by step through the process, he was able to better manage expectations from Stakeholders and customers. He admitted that he didn’t think the product would have ever been finished if he had stayed on his original course, and years of work and millions of dollars would have been wasted.
As a Coach, it’s important to make sure that the ART is delivering incremental value. While the Product Manager is responsible for identifying it, many are new to the role and are trying desperately to understand how to work in this type of environment without big upfront and often sequential design. Any help you can provide in helping them see a simple path through the system will be appreciated.
System Architect
The System Architect is the technical authority for the ART. The System Architect is responsible for not only the overall Architecture but also for ensuring that consistent engineering practices are followed.
The System Architect’s background and knowledge will vary widely, depending on the solution being built. For example, if we are building a software solution, we would likely want a System Architect that has been a Software Engineer. However, if we are building a submarine, we would likely want a System Architect with a background in Aeronautic Engineering.
The System Architect needs to be able to effectively communicate and work well with others. The following diagram shows some of the key collaborations the System Architect engages in:
Figure 6.5 – System Architect responsibility areas (© Scaled Agile, Inc.)
Finding and identifying a System Architect can be significantly challenging, given the breadth and scope that is expected to be known. Typically, the individual has significant industry experience, is organized and thoughtful, personable, and approachable, and understands the importance of teamwork and collaboration.
The System Architect drives the how for the entire system, whereas Product Management articulates what needs to be built. The System Architect needs to ensure that the underlying architecture and infrastructure simplify and accelerate building and delivering Features with Built-In Quality.
The Responsibilities of the System Architect
The five areas of responsibility for the System Architect are outlined in Figure 6.6:
Figure 6.6 – System Architect areas of responsibility (© Scaled Agile, Inc.)
The System Architect has a significant role in the ART and isn’t alone in its execution. The System Architect relies heavily on the System Team to deliver the tools necessary for the ART to deliver successfully. They also rely on the teams to ensure they are following Built-In Quality practices, minimizing technical debt, and balancing the delivery of Enablers and Features.
Pro tip
I have come across several clients who have attempted to map their job titles and roles together, particularly when it comes to architecture. I strongly encourage avoiding this practice and leveraging the dual operating model. I encourage you to ensure that your organization understands that the System Architect role, in particular, is greater than what an Architect is expected to execute.
Now that we understand the roles and responsibilities of the Troika, let’s take a look at the System Team and the role it plays on the ART.
Do we need a System Team?
Simple answer – yes, even though you will hear or read about, and maybe even experience, a successful ART without one. In these instances, there is likely a System Team, but it isn’t formally defined, it isn’t aligned with the ART, the infrastructure has already been built, or even there may be an entire ART dedicated to building and maintaining the infrastructure your ART uses.
What is a System Team?
A System Team is simply a group of individuals that works to support teams in delivering their systems. The larger the system that’s being supported, the greater the need for the System Team. They create and manage the infrastructure and environments that support continuous integration, automated builds, and automated testing: the DevOps pipeline.
Figure 6.7 highlights five primary areas of responsibility for the System Team:
Figure 6.7 – System Team responsibility areas (© Scaled Agile, Inc.)
Based on these responsibilities, we can see how critical the System Team is in helping the ART continuously deliver value.
While an ART can theoretically function without a System Team, there is a good chance that someone or multiple individuals on the ART are executing the role. Many ARTs believe that the infrastructure already in place will be sufficient to support the ART and do not invest in a System Team. Alternatively, there is a perception that the team members on the teams have the skills and abilities to build the pipeline and ensure its security and stability.
Establishing an official System Team shows dedication to and support for the ART and its ability to deliver. Keep in mind that it is not the System Team’s responsibility to be fully responsible for solution integration. It’s a partnership with the teams.
The System Team allows an ART to mature and to share knowledge across the teams and other ARTs. As the infrastructure is built, the System Team ensures that knowledge is shared, and it becomes the established way of working.
Pro tip
In the SAFe® ART Readiness Workbook, one of the items is “Has the System Team been identified and formed?” In the first ART I launched, I was unable to answer that question with a yes. Now, I won’t launch an ART without one.
As you can see, the System Team brings a lot of benefits to the ART. They have key responsibilities that are often lost in the day-to-day work and execution of the teams and ART. Do you have to have a System Team? No. However, the ability of the ART to deliver will be impacted, the maturity of the ART will slow, and the visibility of the work to effectively and efficiently keep the Train running down the tracks will be hidden and will eventually cause the ART to crash.
Pro tip
As a Coach, I initially didn’t think the System Team was that important when launching an ART, however after launching a few ARTs that struggled until we formalized this team, it is now one of the first teams I identify.
I encourage you to ensure you know who is doing this work even if you don’t have a formal team identified.
Summary
In this chapter, we looked at the structure and key attributes to consider when standing up an ART. We learned why it’s important to align your ARTs with your Value Streams and some challenges you may encounter if you don’t. We discussed the critical roles in the ART, including the RTE, Product Management, and System Architect, and that these three roles collectively comprise the Troika. We learned why the Troika is important and its impact on the ART. Lastly, we learned how critical a System Team is to our ART and why we shouldn’t overlook this team.
Further reading
To learn more about the topics that were covered in this chapter, take a look at the following resources:
- Conway’s Law: https://en.wikipedia.org/wiki/Conway%27s_law
- The Agile Release Train: https://www.scaledagileframework.com/agile-release-train/
- Release Train Engineer: https://www.scaledagileframework.com/release-train-engineer/
- Product Management: https://www.scaledagileframework.com/product-management/
- System Architect: https://www.scaledagileframework.com/system-architect/
- System Team: https://www.scaledagileframework.com/system-team/
- Operational Value Streams: https://www.scaledagileframework.com/operational-value-streams/
- Development Value Streams: https://www.scaledagileframework.com/development-value-streams/