Organizational Changes

Organizations can make Continuous Delivery easier by making some changes to the shape, size, and responsibilities of teams. One of the most effective changes is to align software delivery teams with a product or service: the team works on a program of changes and features for one or more product and service, but only that team. Instead of projects we use programs of work that have a clear and consistent timeline over many months and years.

This alignment with products and services typically helps to work around the problems of Conway’s law—the tendency to push the software we build into shapes that resemble the communication structures in the organization. This alignment with services works well with recent practices such as DevOps, where responsibility for the build and operation of software systems is shared between several teams with complementary skills, as shown in Figures 7-1 and 7-2.

Figure 7-1. Function-focused teams produce a need for handoffs

Figure 7-2. Cross-functional product-focused teams

To aid Continuous Delivery, development tasks need to be approached in a manner that promotes continuous flow and short feedback loops. There are various agile methodologies aimed toward these goals such as Scrum, Kanban, and XP. Any of these methodologies may work for your team—it is something you will need to experiment with. The key factors are to keep work in progress (WIP) limits low, highlight any blocked tasks, and fully complete tasks to “release” before moving on to another task. It is essential to keep the work “flowing.” The Theory of Constraints, as described in The Goal [GoldrattCox] and more recently The Phoenix Project [Kim], is an approach for highlighting bottlenecks in the development process and as such, in preventing a buildup of incomplete changes from becoming stalled.

Organizations should expect to allocate the equivalent of around one person on build and deployment activities per development team of 8–10 people (what Amazon has referred to as a two-pizza team). Build and deployment are so fundamental to Continuous Delivery that they need strong, ongoing investment if Continuous Delivery is to be successful.

Testing for Continuous Delivery looks very different from traditional approaches. Instead of separate QA or Release Testing teams, we embed testers within product/service teams, so that the valuable questioning mindset of the tester can complement the optimistic mindset of the developer, and we build quality in.

Architectural Changes (SOA/Microservices)

The fast rate of release in a Continuous Delivery environment promotes certain architectural approaches of a platform. Releasing often, with small changes, is extremely difficult to do with a monolithic architecture.

We recommend rearranging the architecture into smaller, focused, domain-specific applications—a service-oriented architecture (SOA) or even one composed of microservices. These architectures come with their own challenges: data consistency, service availability, and partition tolerance (known as the CAP theorem). The benefit of a distributed system is that it allows for each of the components to change and evolve with only loose coupling between them.

If the contracts between each component are well defined and stable, this approach allows for individual components to be developing and releasing change at the same time. The overall rate of change will greatly increase over that of a monolith where change has to be highly coordinated.

Operational Features

Many organizations treat functional and nonfunctional requirements separately, with nonfunctional aspects often deprioritized or given to IT Operations teams to handle. This approach is not suitable for Continuous Delivery, because Continuous Delivery needs high-quality, well-tested, production-ready software, not just a set of user-visible features that seem to work on a developer’s laptop or in a UAT (User Acceptance Test[ing]) environment.

There are many invisible considerations that we need to address for Continuous Delivery, including performance, resilience, deployability, security, and availability. The ability to test new code—testability—becomes an important consideration for Continuous Delivery; we generally prefer code that is more easily testable than code that is, say, more clever but less testable.

There are several useful techniques for addressing operational concerns. Rather than separating functional and nonfunctional requirements, organizations should prioritize these requirements together in a single backlog, pairing up a Product Owner and a Tech Lead if necessary. We can also identify an IT Operations role as a secondary user type (or persona) and write User Stories for them: “As an IT Ops person, I need an automated way to check the runtime status of a component so that I can diagnose health problems within 60 seconds,” for instance. Some organizations place a kind of “tax” on product budgets—usually around 20%–30% of the total product budget—which is used for operational concerns. The tax is essentially an honest way of accounting for the operational costs incurred with modern, complex software systems; either we pay for them up front with a slice of the product budget, or the organization pays for them in unplanned downtime, security breaches, or rework after failed technical testing.

Summary

Continuous Delivery requires changes not only to the technologies and techniques we use for building and deploying software but also to the way in which we structure our teams, our software architecture, and our approach to operational concerns. These tricky aspects are essential for a sustainable approach to Continuous Delivery that will provide lasting results regardless of the tooling and technologies used.