Chapter 11. Technical Debt Credit Check
You want to execute a deep analysis of your system and formulate a strategy for managing your technical debt. Beginning with a quick sanity check of the business goals against the system architecture, development practices, and organizational context will provide guidance for successfully executing that deeper analysis and determining actionable outcomes. In this chapter, we introduce a technique for assessing the context and state of your software development project to reveal the causes of your debt.
Identifying Causes: Technical Debt Credit Check
How do you begin to manage a complicated situation? Consider this scenario: You come home and find out your living room is flooded. What do you do first? Do you think about the best plumber to call? Do you mop the floor? Do you call your insurance agent? Or do you quickly look around to assess the situation, see if the water is continuing to come in, shut off the main water valve, move your belongings out of harm’s way, and then figure out the cause, the source of the water?
Likewise, a quick check of a project and the software and system under development may reveal technical risks in the business vision, architecture, organization, and development practices that can potentially inject technical debt into the system. You can use these findings to define criteria for measuring technical debt and select techniques and tools that will help you measure against those criteria. You will find this information useful if you plan to conduct an overall technical debt analysis of your system to fully characterize its current state of technical debt.
The goal of the Technical Debt Credit Check is to identify the root causes of a system’s existing technical debt and determine whether the debt will continue to grow. Understanding the causes of debt is essential for selecting the appropriate management practices and removing the debt. In Chapter 10, “What Causes Technical Debt?” we looked at common causes of technical debt. The Technical Debt Credit Check helps teams understand what might be causing their debt, especially if it is at a chronic level. This simple questioning technique enables teams to quickly review their business vision, the organization’s capacity to support that vision, and the software development artifacts and practices. For the flooded living room, such a check would help you determine whether the pipes leaked, the dishwasher overflowed, or someone left a faucet on. Once you find the source, you can stop the flow of water and look for damage in other areas of the house that might not be as visible as the water in the living room.
In the next few subsections, we describe the purpose of this technique, who participates and when, what types of input you need, the steps to proceed, and the outcome.
Purpose
The Technical Debt Credit Check is a systematic approach to navigate through the context and state of a software development project, using four focus areas that are worthy of attention. By reviewing key criteria, an organization can quickly identify potential causes at risk for creating technical debt and that need further analysis. This initial technique is especially useful when an organization is dealing with the consequences of unintentional technical debt. In Chapters 5, “Technical Debt and the Source Code,” 6, “Technical Debt and Architecture,” and 7, “Technical Debt and Production,” we described how to choose appropriate analysis approaches to further clarify the root causes and trace technical debt to the development artifacts associated with code, architecture, and deployment.
Who Is Involved?
This technique identifies the potential causes of technical debt from the perspective of the development team and project management. Developers have firsthand knowledge of the development artifacts associated with technical debt and its symptoms, and managers grasp the consequences for cost and value. A small group of two or three members of the project team will act as analysts and interview the key stakeholders of the project, focusing on areas of the business vision, architecture, development, and organization.
When Can You Conduct?
You can use this technique in two ways. An organization or a team may sense that technical debt is building up but find it hard to begin addressing it systematically. In such a case, you can use the technique as an intervention to bring awareness of technical debt into the organization. You can also use the technique as an ongoing activity that is part of the project’s process for continuous improvement. In this case, you establish a baseline of areas that are most likely to contribute to technical debt and allocate analysis and management resources accordingly. Once the baseline is established, you can use the criteria to investigate causes periodically to keep technical debt under control.
Inputs
The inputs to this technique are the context and state of the software development project, focusing on the business vision, architecture, development practices, and organization. Focusing on these areas will help the team move quickly from overgeneralized causes that may express any project’s struggles (summarized in Chapter 10) to project-specific causes that the team can use to identify concrete technical debt items. The inputs may be found in multiple artifacts as well as in the heads of developers and key stakeholders and in the tribal knowledge of the organization.
Steps
Here is a step-by-step description of how to conduct a Technical Debt Credit Check:
In collaboration with the project decision makers, select key stakeholders to interview. At a minimum, select junior and senior developers, the architect, the project manager, and key decision makers.
Discuss the state of the project, focusing on the business vision, architecture, development, and organization. Ask questions targeted to uncover the common causes of technical debt, as detailed later in this chapter.
Consolidate issues across the key focus areas to identify causes that lead to risks related to the business goals, the architecting activities, the development practices, and how people are supported by the organization. Issues may be similar or may overlap.
Present the results to all the stakeholders and the key decision makers.
Guide the stakeholders in prioritizing the identified causes. Estimate the probability of occurrence and the potential impact of each risk. Rank them according to their potential risk of triggering technical debt—high, medium, or low. Express the resulting list as typical risk areas but focus on technical debt, as in these templates:
If <bad thing that may happen>, then <negative consequence> might result.
<Factual statement of existing situation> may lead to <negative consequence>.
Output
The output is a scorecard that includes a list of causes of technical debt and the impact rating of each as high, medium, or low.
Four Focus Areas for Understanding the State of a Project
We suggest that you initially focus on four key areas to understand the context and state of a project. These four areas will help you filter the myriad causes to a handful of them to develop a simple, actionable strategy for technical debt triage.
Business Vision
Through a clear vision of the business goals for the system, a project team will understand the desired system qualities, the desired software development state, and the consequences of choices that developers make when diverging from that state. Without this clear vision, a system can suffer many unintended consequences that result in technical debt. Key criteria to investigate to ensure that the development effort is aligned with the business vision include the following:
Are business goals clear, and do they reflect stakeholders’ concerns?
Are success strategies defined and clearly communicated (for example, roadmaps, product portfolios, key timelines)?
Is funding secured, and are there related resource priorities that could affect the project?
Does the product owner understand the dynamics of the business environment and changing market opportunities?
Are consequences of key business decisions for product design and development clear?
Has the development team established effective communication channels with the customer? And timely feedback cycles?
The focus on business vision will help you identify business-related causes (as discussed in Chapter 10), such as the magnitude of time and cost pressures, alignment of business goals, and clarity of requirements.
Architecture
Architecting activities that balance the short-term and long-term technical goals of a project must be integrated into the software development lifecycle to strategically manage technical debt. The view that these activities happen sequentially often creates silos, architecture conformance issues, and unexpected rework costs in later stages of the development effort. Teams must make architectural decisions with consideration for business goals, organizational needs, and the desired state of development. Key criteria to investigate include the following:
Are architecturally significant requirements defined, tied to business goals, and communicated clearly across the business and technical stakeholders?
Is evidence provided that the architecture satisfies key requirements?
Are there known architectural issues, and are they tracked and managed?
Is the timeline of key architectural decisions clear, considering both short-term and long-term business goals that the architecture needs to support?
Is the impact of the changes in technology and their limitations clear?
Are key build and integration, test, and deployment scenarios clear, well developed, and utilized in a timely manner?
Identifying causes of debt related to the architecture will uncover causes related to the context of the project, such as technology change, business shift, or market evolution. It will also give hints about where the most critical technical debt may reside in the system. In addition, you may uncover process-related causes—not only processes related to architecture but also processes related to documentation and software development. Process causes are important because they indicate how well your processes will guide the team to manage and service technical debt.
Development
The bottom line of any software engineering project is the quality of the running system. It is critical to align the development practices with the business goals and architecture to avoid unintentional debt. Investigating the following criteria will help uncover potential risks related to development and its processes and tools:
Is the development infrastructure in place and aligned with the architecture?
Are necessary quality control methods available and used (for example, code reviews, inspections, testing, continuous integration, deployment practices)?
Does the development team have appropriate tools and use them effectively? Is training provided when needed?
Is an environment in place to measure and monitor the implemented system quality and “done” criteria?
Has the development team considered code maintenance and evolution?
Does the team understand, embrace, and follow the established software development processes and practices?
Organization
Any successful organization operates within a culture and established procedures. When an organization’s underlying culture and processes do not support its people and embrace change, technical debt creeps in. Key criteria to investigate include the following:
Does the organizational structure enable collaboration? Do the development team, project management, and architects effectively support each other?
Are necessary procedures and technology in place to respond to change?
Has the organization determined the impact of cost of delay and rework and decided how to manage trade-offs?
Has the organization acknowledged the impact of uncertainty on the project?
Has the organization provided training for skills needed to succeed in the project?
Has the organization provided the team with sufficient resources?
Is there a procedure to bring new team members up to speed with the project?
Have teams and team members established clear communication channels?
Diagnosing the Causes of Technical Debt in Phoebe
The example we gave in Chapter 10 from the Phoebe project showed that technology change was the cause of an architectural issue:
The open-source web services stack that we rely on went through several versions, but we did not upgrade. Our customers require new features that we cannot support if we do not upgrade soon.
Figure 11.1 shows the scorecard for causes of technical debt in the Phoebe project. We recommend rating the outcomes as red, yellow, and green, where red indicates that the issue area is causing technical debt, yellow indicates that the area is a potential cause of technical debt if not managed better, and green indicates that the area is being managed adequately. A red rating implies that the answers to the questions under that focus area were mostly negative or insufficient. Phoebe clearly needs to better manage its short-term and long-term architectural issues.
Diagnosing the Causes of Technical Debt in Tethys
Let us look in detail at Tethys, an organization that needs to focus on aligning its business goals and organizational processes. As Tethys grew into the global giant it is today, management decided to separate the responsibilities of development and quality assurance. The development teams operated in an iterative and incremental delivery tempo, and the quality assurance team followed a waterfall approach to the software development lifecycle. In safety-critical and avionics environments, where products must conform to industry and safety standards, such over-the-wall handoff between development and quality assurance is not uncommon.
However, the delivery schedule of the development teams did not align with the quality assurance teams’ expectations and schedule. The development teams delivered features in increments. The quality assurance team did not test feature increments even though other features relied on them, waiting instead to test each entire feature once it was complete. The inevitable consequence of this practice was that quality assurance found defects, and developers’ time became consumed with fixing them. While the project manager prioritized new features above all other tasks, new feature development started slipping, and time pressure to deliver became the development team’s top priority; the development team had to navigate conflicting priorities of fixing defects and developing new features. In these ways, misalignment of business goals and organizational processes created substantial roadblocks for Team Tethys.
In addition, another organizational issue had causes rooted in inexperienced teams, an area that contributes many causes of potential technical debt across the software industry. One Tethys developer reflected on this issue:
We have a very high turnover rate, but we do not allocate the time to bring the new hires up to speed with the system and our development practices. New hires inject a lot of defects because we do not onboard them properly. When more experienced staff members mentor new hires, they succeed, and we do not see the chaos, but often no one takes enough time to do this, and then issues go unnoticed. This lack of training will eventually slow down our velocity, which is already showing signs of the problem.
The team lead of Tethys negotiated with his manager and customer to postpone new features as the inexperience of junior members was becoming more problematic with each iteration. While the team fixed some of the immediate issues caused by these defects, he conducted a Technical Debt Credit Check and reported the results in the scorecard shown in Figure 11.2.
A closer look at the Tethys business vision revealed a misalignment of business goals. The project team did not understand the product-line opportunity. Whilethe long-term goal was to serve multiple markets with the same product, the short-term goal was to serve a pressing time-to-market requirement. The development teams went out of their way to create a general architecture, and they missed the immediate product-specific delivery needs, further adding time and cost pressure.
Tethys began its journey to the market with significant risks that the project team would inject technical debt into the product because of its confusion about short-term goals and the business vision. The goal of creating a solution that envisioned all potential variations of the product resulted in an over-parameterized architecture. In an effort to create an infrastructure robust enough to handle the natural evolution of the products and product line, the team added unnecessary complexity for the immediate customer need. Both the overgeneralized architecture and unnecessary complexity created several technical debt items as development progressed. The team got lost within the variation parameters, so many of the features they implemented were incomplete. Alternatively, focusing exclusively on the short-term goals of the immediate customer would have introduced a different set of issues. Getting clarity on the trade-offs would have helped team members recognize where they would need to take on technical debt with intention so that they could manage it strategically. The Tethys project would have taken on technical debt either way, but it missed an opportunity to take on the debt strategically and not only acquired the wrong type of debt but also did not recognize it until it almost got the project canceled.
The organizational structure of Tethys provides clues about how a number of causes related to process contributed to the accumulating technical debt. Tethys was not able to align the multiple processes of the iterative and waterfall models across the distributed development and quality assurance teams, which operated at different tempos due to the challenges of fulfilling compliance requirements in the safety-critical domain.
The Tethys project used three different cycles: annual releases to the customer, quarterly testing performed by a quality assurance team, and monthly sprints conducted by the development team. The goals of each cycle were different, yet they had important dependencies. For example, by the time the quarterly testing found issues with product features in a given release, the development team had already implemented three other releases on top of the system, increasing its size and complexity. This made it harder for the team to locate issues, so they spent an extensive amount of time in bug-fixing mode. Consequently, by the time the developers realized that they could not make progress while using three unaligned cycles, it was too late to redesign the overly complex architecture that resulted in unmaintainable and buggy code.
After conducting the Technical Debt Credit Check, team members realized that their misaligned business goals created the overly complex architecture. They conducted an analysis of their codebase, using some static analysis tools, focusing on security (as discussed in Chapter 5). To understand the impact of that debt, they conducted an architecture review (as discussed in Chapter 6). Consequently, the team decided to take the following corrective actions immediately to reduce the debt and avoid its further accumulation: Reduce the number of variant parameterizations in the architecture, add guidelines for architecture conformance, and have everyone work from the same architecture. There were other consequences of the Technical Debt Credit Check, such as replanning the release cycles to better align the development and testing cycles and revisiting the testing strategy completely (as discussed in Chapter 7).
Mapping the events onto the technical debt timeline as shown in Figure 11.3 allowed the Tethys team to assess the consequences of its debt. Team members continued to mitigate risk by remediating the debt, as discussed in Chapter 9, “Servicing the Technical Debt.” They decided to stop delivering new features for at least a quarter until they repaid some of the debt. Distinguishing the causes of their debt and the current debt that they needed to fix allowed them to recognize that if they kept fixing defects, they would never get ahead of the problem. They needed to correct the course of their product-line architecture.
What Can You Do Today?
Teams that do not follow established software engineering practices will take on reckless and unintentional technical debt. In this chapter, we introduced a technique to help you identify where you may be diverging from established practices and introducing technical debt.
With the right stakeholders in the room and good facilitation skills, you can conduct your own Technical Debt Credit Check and create a scorecard indicating the causes that contribute most to your technical debt accumulation. Then you can begin developing a plan to manage your debt strategically.
For Further Reading
Technical risk assessment is a routine practice in many organizations. The Technical Debt Credit Check we describe is inspired by these approaches, but it is meant to provide a lightweight approach to assessing technical debt risks. The Architecture Tradeoff Analysis Method by the Software Engineering Institute (Bass et al. 2012), for example, similarly walks through the architecture of a system to uncover technical risks against business goals and architecturally significant requirements.
The guidelines that the Agile Alliance Technical Debt Initiative has developed for executives, managers, and developers summarize code quality rules that, when violated, generate technical debt. In particular, they propose an Agile Alliance Debt Analysis Model (A2DAM) (Fayolle et al. 2018).