This is the dimension most people seem to mean when they talk about scaling. Every medium-sized company I've visited has fond memories of how things used to be. Problems would just get solved. At some point they realized that the way problems were solved when there were two programmers no longer worked. Their solutions “didn't scale.” While it is true that fifty developers can't act like two, the rigid controls and sign-offs frequently instituted aren't the only solution.

When faced with a big problem I work in three steps:

Following this, when faced with the apparent need for a large team, perhaps the problem really could be solved by a small team. I've seen organizations grow from fifty to three hundred and make plans to grow to two thousand, adding to their problems and losing overall throughput every step of the way. Addicted to scaling in number of employees, these organizations don't seem to want to believe that their problems would be better solved by those fifty original developers.

If just using a smaller team doesn't work, turn the big programming problem into several smaller problems, each solvable by a small team. First solve a small part of the problem with a small team. Then divide the system along its natural fracture lines and begin working on it with a few teams. Partitioning introduces the risk that the pieces won't fit on integration, so integrate frequently to reconcile differing assumptions between teams. This is a conquer-and-divide strategy instead of a divide-and-conquer strategy. Sabre Airline Solutions, profiled in the next chapter, uses this strategy extensively.

The goal of conquer-and-divide is to have teams that can each be managed as if they are the only team to limit coordination costs. Even so, the whole system needs to be integrated frequently. The occasional exceptions to this illusion of independence are managed as exceptions. If the exceptions become the norm and the teams have to spend too much time coordinating, look to the system to see if there are ways of restructuring it to return the teams to independence. Only if this fails is the overhead of large-project management appropriate.

In summary, faced with the apparent need for a large team, first ask if a small team can solve the problem. If that doesn't work, begin the project with a small team, then split the work among autonomous teams.

I'm often asked how to account for large investments in XP projects. For example, I've been asked several times whether XP-style development is an expense or a capital investment. Companies that like to expense development can justify XP as ongoing maintenance to a deployed program. Companies that account for most software development as a capital investment can use a quarterly or annual cycle to approve large amounts of development aimed at specific problems, even if the precise scope of the projects is not specified in detail in advance.

The problem lies in how accounting handles software development, not in XP itself. Blindly applying accounting models from the world of factories and widgets to an activity as different as software development inevitably creates distortions in the accounts. There is interesting work to be done outside the scope of XP in rethinking a mutually beneficial relationship between accounting and software development.

If you are starting large-scale software development XP-style, find an ally in finance early on to help you navigate these issues. Each company seems to account for software a little differently.

How do you apply XP in part of an organization, when most of the organization isn't changing? While the team should quickly begin to create both more and more accurate information, forcing that information on unwilling listeners creates enemies that the team needs as friends. The goal is neither to hide the new workings of the team nor to force others to change. Be sure to maintain communication with the rest of the organization in the forms they are accustomed to.

This is an area where an XP team can benefit from a skilled project manager. If the big monthly staff meeting expects slides in a certain format, then that's exactly what the XP project manager prepares. The project manager presents the information in a form the organization can absorb. The story cards on the wall are still “the truth”. Anyone who wants to learn to read them is welcome to come in, look at them, and ask questions. The project manager makes sure that the organization's expectations are met. This can be a challenge since what is going on inside the XP team is so different from what goes on inside other teams. Respect the others in the organization. Don't push your newfound knowledge and power on others for your own benefit.

Meeting organizations' expectations and at the same time maintaining what is good about XP sometimes requires creativity. At one client, each project was required to prepare a detailed quarterly plan. This seemed to be incompatible with XP and negotiating scope weekly. The boss's boss, however, was wise enough to discover the purpose of the quarterly plan and find out when it would be read. The plans were only read at a quarterly executive review at the end of the quarter. The plans were compared with what actually happened to see if the teams had acted responsibly.

On the XP team, the project manager came around every Friday and asked each team member what he had done that week. She entered this information in the quarterly plan format. At the quarterly review, the team's plan was observed to contain the most precise estimates in the organization.

No one was trying to lie with this process. The whole thing took place above board. The whole management chain knew what was happening. The team satisfied the letter of the organization's expectations by being in the right format and it satisfied the spirit of the quarterly plan by making sure it spent its time responsibly.

Long-running XP projects work well because the tests prevent many of the common maintenance mistakes and tell the story of the development process. The simplest case of scaling in time is if the team maintains continuity throughout the project. Then automated testing and incremental design serve to keep the system alive and capable of further growth. A team I coached ten years ago using these techniques has been steadily adding functionality ever since. Defects are rare. Progress is unspectacular but steady. Progress doesn't need to be spectacular to grow a big, sophisticated system in ten years, even with a small team.

Projects that start and stop frequently, with the team scattered at each shutdown, are more difficult to maintain over time. In this case, XP teams often write a “Rosetta Stone” document before shutting down the project. This brief guide to future maintainers tells how to run the build-and-test process and points to interesting starting places from which to learn the system. The tests included in the build prevent maintainers from falling into pits while they learn their way around the system.

XP is ideally suited to projects requiring the close cooperation of specialists. One challenge at the beginning of such projects is getting everyone to work in concert while learning a bit about each others' specialities. For example, once I worked on a life insurance project. The actuary was patient with me as I began to learn enough actuarial math to pair with him. After a month, I was catching stupid mistakes. After several months, I was even helpful sometimes. I never became an actuary, but the resulting system (and team) was much stronger than if the actuary worked on his little corner of the system while I worked alone on the user interface.

Sometimes systems grow big and complicated, out of proportion to the problem they solve. The challenge is to stop making the problem worse. It is difficult for a struggling team to keep going when every defect fixed creates three more. XP can help.

One client began by getting the build process under control. The team improved the build so instead of taking 24 hours on a dedicated machine with lots of manual intervention, the build took an hour and could run completely automatically on any machine. Then, the team instituted stories and a story board so everyone knew who was working on what and how long they were taking. After two years of steady improvement the team reduced costs 60%, going from seventy engineers to twenty; reduced the time to fix defects 66%; and reduced the time to release for major and minor point releases by 75%, from ten weeks to two weeks. Once the team had stopped digging itself in deeper, it began to climb out by eliminating excess complexity while also fixing defects.

The XP strategy for dealing with excess complexity is always the same: chip away at the complexity while continuing to deliver. Brighten the corner where you are. If you are fixing a defect in an area, clean up while you are there. One objection is that this “extra” cleanup takes too long. The team is likely wasting time on interruptions to fix defects. Cleaning up helps reduce the overhead of work. Visible planning can make it easier for every one to see where the time is already going so it is easier to accept the estimates necessary to do the job right.

How do you use XP in a safety- or security-critical project? Some of the rules change because the number one value becomes safety or security. As a hospital software team I met put it, “If we make a mistake, babies die.” More than XP is needed in life-critical situations.

A system isn't certifiably secure unless it has been built with a set of security principles in mind and has been audited by a security expert. While compatible with XP, these practices have to be incorporated into the team's daily work. For example, refactorings have to preserve the security of the system as well as its functionality.

Avionics and medical systems are audited before they are allowed to be deployed. XP's principle of flow suggests that auditing should not be a phase at the end of a project. Auditing should happen early and often. The instructions to auditors such as DO-178B explicitly allow software development lifecycles other than a strict waterfall. Here is an example from the Food and Drug Adminstration's “Guidance for FDA Reviewers and Industry Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices”:

Building an ongoing relationship with your auditor improves your chances of a successful audit.

Traceability, the ability to link what has changed in a system to why it changed, is built into XP, although the information isn't routinely recorded. The only change to implement traceability is to make a physical record of this information. If I am changing this line of code, it is because I wrote that test which is a part of that system-level test which came from that story which was scheduled May 24 and was ready to deploy on May 28. Your auditor will tell you what format to use in saving this information.

With awareness and appropriate adaptations, XP does scale. Some problems can be simplified to be easily handled by a small XP team. For others, XP must be augmented. The basic values and principles apply at all scales. The practices can be modified to suit your situation.