People develop software. This simple, inescapable fact invalidates most of the available methodological advice. Often, software development doesn't meet human needs, acknowledge human frailty, and leverage human strength. Acting like software isn't written by people exacts a high cost on participants, their humanity ground away by an inhumane process that doesn't acknowledge their needs. This isn't good for business either, with the costs and disruption of high turnover and missed opportunities for creative action.

What do people need to be good developers?

I chose practices for XP because they meet both business and personal needs. There are other human needs; such as rest, exercise, and socialization; that don't need to be met in the work environment. Time away from the team gives each individual more energy and perspective to bring back to the team. Limiting work hours allows time for these other human needs and enhances each person's contribution while he is with the team.

Part of the challenge of team software development is balancing the needs of the individual with the needs of the team. The team's needs may meet your own long-term individual goals, so are worth some amount of sacrifice. Always sacrificing your own needs for the team's doesn't work. If I need privacy, I am responsible for finding a way to get my need met in a way that doesn't hurt the team. The magic of great teams is that after the team members develop trust they find that they are free to be more themselves as a result of their work together.

While intimacy feels great, work is still work. Private details of life confuse team communication. I talked to one team in which a member, once he got comfortable on the team, talked about private details of his life every morning. No one else was comfortable hearing about his private life but they didn't know how to confront him. Eventually, the senior team member took him aside and asked him to keep private matters private.

I try to separate my life into private matters that I only discuss with my spouse, personal matters that I discuss with those who have earned my trust, and public matters that I don't mind talking about with anyone. Figuring out which is which is not a simple matter, nor is it simple to figure out who to trust. The rewards of this separation when it works well are effective communication on the job and relationships that are valuable in all aspects of my life.

Somebody has to pay for all this. Software development that doesn't acknowledge economics risks the hollow victory of a “technical success”. Make sure what you are doing has business value, meets business goals, and serves business needs. For example, solving the highest priority business need first maximizes the value of the project.

Two aspects of economics that affect software development are the time value of money and the option value of systems and teams. The time value of money says that a dollar today is worth more than a dollar tomorrow. Software development is more valuable when it earns money sooner and spends money later. Incremental design explicitly defers design investment until the last responsible moment in an effort to spend money later. Pay-per-use provides a way of realizing revenue from features as soon as they are deployed.

Another source of economic value in software development is its value as options for the future. If I can redeploy my media scheduling program for a variety of scheduling-related tasks, it is much more valuable than if it can only be used for its originally intended purpose. All the practices are intended to enhance the option value of both the software and the team while keeping in mind the time value of money by not investing in speculative flexibility.

Every activity should benefit all concerned. Mutual benefit is the most important XP principle and the most difficult to adhere to. There are always solutions to any problem that cost one person while benefitting another. When the situation is desperate, these solutions seem attractive. They are always a net loss, however, because the ill will they create tears down relationships that we need to value. The computer business is really a people business and maintaining working relationships is important.

Extensive internal documentation of software is an example of a practice that violates mutual benefit. I am supposed to slow down my development considerably so some unknown person in a potential future will have an easier time maintaining this code. I can see a possible benefit to the future person should the documentation still happen to be valid, but no benefit now.

XP solves the communication-with-the-future problem in mutually beneficial ways:

If you want people to take your advice, you need to solve more problems than you create. Mutual benefit in XP is searching for practices that benefit me now, me later, and my customer as well. Win-win-win practices are easier to sell because they relieve some immediate pain. For example, someone wrestling with a tough defect is ready to learn test-first programming. When it benefits me now, it is easier to accept doing something to help others both now and in the future.

One day I went walking along the Sardinian coast. I saw a little tide pool, maybe two feet across, with the shape outlined in Figure 2. I looked up and noticed that the bay I was walking around, maybe a mile across, had roughly the same shape. “What a great example of the fractal nature of geology,” I thought to myself. This drawing is actually a tracing of a map of the whole northwest corner of Sardinia. When nature finds a shape that works, she uses it everywhere she can.

Figure 2. Naturally occurring shape

The same principle applies to software development: try copying the structure of one solution into a new context, even at different scales. For example, the basic rhythm of development is that you write a test that fails and then you make it work. The rhythm operates at all different scales. In a quarter, you list the themes you want to address and then you address them with stories. In a week, you list the stories you want to address, write tests expressing the stories, then make them work. In a few hours, you list the tests you know you need to write, then write a test, make it work, write another test, and make them both work until the list is done.

Self-similarity isn't the only principle at work in software development. Just because you copy a structure that works in one context doesn't mean it will work in another. It is a good place to start, though. Likewise, just because a solution is unique doesn't mean it's bad. The situation may really call for a unique solution.

In the first edition of Extreme Programming Explained, my advice for the weekly cycle was much more like a waterfall: write some code, then test it to make sure it works. I should have paid attention to self-similarity. Having the system-level tests before you begin implementation simplifies design, reduces stress, and improves feedback.

In software development, “perfect” is a verb, not an adjective. There is no perfect process. There is no perfect design. There are no perfect stories. You can, however, perfect your process, your design, and your stories.

“Best is the enemy of good enough” suggests that mediocrity is preferable to waiting. This phrase misses the point of XP, which is excellence in software development through improvement. The cycle is to do the best you can today, striving for the awareness and understanding necessary to do better tomorrow. It doesn't mean waiting for perfection in order to begin.

In translating values to practices, the principle of improvement shows in practices that get an activity started right away but refine the results over time. The quarterly cycle is an expression of the possibility of improving long-term plans in the light of experience. Incremental design puts improvement to work by refining the design of the system. The actual design will never be a perfect reflection of the ideal, but you can strive daily to bring the two closer.

The history of software development technology shows us gradually eliminating wasted effort. For example, symbolic assemblers eliminated the wasteful tedium of translating machine instructions into physical bit encodings; “automatic programming” then eliminated the wasteful tedium of translating an abstract description of a program into assembly language; and so on up through automatic storage deallocation.

While our improved technology has eliminated waste, our increased rigidity and specialized social structures in development organizations are increasingly wasteful. The key to improvement is reconciling the two, using newfound technological efficiency to enable new, more effective social relationships. Put improvement to work by not waiting for perfection. Find a starting place, get started, and improve from there.

Software development teams where everyone is alike, while comfortable, are not effective. Teams need to bring together a variety of skills, attitudes, and perspectives to see problems and pitfalls, to think of multiple ways to solve problems, and to implement the solutions. Teams need diversity.

Conflict is the inevitable companion of diversity. Not conflict in the “we hate each other and we just can't make progress” sense, but in the “there are two ways to solve this” sense. How do you choose?

Two ideas about a design present an opportunity, not a problem. The principle of diversity suggests that the programmers should work together on the problem and both opinions should be valued.

What if the team isn't good at conflict? Every team has conflict. The question is whether they resolve it productively. Respecting others and maintaining myself smooths communication in times of stress.

Diversity is expressed in the practice of Whole Team, where you bring together on the team people with a variety of skills and perspectives. The various planning cycles encourage people with different perspectives to interact with the goal of creating the most valuable software possible in the time available.

Good teams don't just do their work, they think about how they are working and why they are working. They analyze why they succeeded or failed. They don't try to hide their mistakes, but expose them and learn from them. No one stumbles into excellence.

The quarterly and weekly cycles include time for team reflection, as do pair programming and continuous integration. But reflection should not be limited to “official” opportunities. Conversation with a spouse or friend, vacation, and non-software-related reading and activities all provide individual opportunities to think about how and why you are working the way you are. Shared meals and coffee breaks provide an informal setting for shared reflection.

Reflection isn't a purely intellectual exercise. You can gain insight by analyzing data, but you can also learn from your gut. The “negative” emotions like fear, anger, and anxiety have long provided cues that something bad was about to happen. It takes effort to listen to what your emotions tell you about your work, but feelings tempered by the intellect are a source of insight.

Reflection can be taken too far. Software development has a long tradition of people so busy thinking about software development they don't have time to develop software. Reflection comes after action. Learning is action reflected. To maximize feedback, reflection in XP teams is mixed with doing.

Flow in software development is delivering a steady flow of valuable software by engaging in all the activities of development simultaneously. The practices of XP are biased towards a continuous flow of activities rather than discrete phases.

Software development has long delivered value in big chunks. “Big Bang” integration reflects this tendency. Many teams make the problem worse by tending to respond to stress by making the chunks of value bigger, from deploying software less frequently to integrating less often. Less feedback makes the problem worse, leading to a tendency for even bigger chunks. The more things are deferred, the larger the chunk, the higher the risk. In contrast, the principle of flow suggests that for improvement, deploy smaller increments of value ever more frequently.

A few trends in software development buck the concept of bigger batches. The daily build, for example, is flow-oriented. However, daily builds are a small step on the road to flow. It is not enough that the software compile and link every day; it should also function correctly every day or, better yet, several times a day.

I visited a team that used to deploy every week. It had more and more problems, until it was taking six days to deploy a week's worth of software. The team chose to deploy every two weeks. This amplified their integration and deployment problems. Any time you move away from flow, resolve to return. Resolve the problems that disrupted your flow and get back to weekly deployment as soon as you can.

Learn to see problems as opportunities for change. This isn't to say there are no problems in software development. However, the attitude of “survival” leads to just enough problem solving to get by. To reach excellence, problems need to turn into opportunities for learning and improvement, not just survival.

You might not know what to do about a problem. You might want more time to think about what to do. Sometimes the desire for more time is a mask worn to protect from the fear of the consequences of getting going. Sometimes, though, patience solves a problem by itself.

Turning problems into opportunities takes place across the development process. It maximizes strengths and minimizes weaknesses. Can't make accurate long-term plans? Fine—have a quarterly cycle during which you refine your long-term plans. A person alone makes too many mistakes? Fine—program in pairs. The practices are effective precisely because they address the enduring problems of people developing software together.

As you begin practicing XP, you will certainly encounter problems. Part of being extreme is consciously choosing to transform each problem into an opportunity: an opportunity for personal growth, deepening relationships, and improved software.

Yes, redundancy. The critical, difficult problems in software development should be solved several different ways. Even if one solution fails utterly, the other solutions will prevent disaster. The cost of the redundancy is more than paid for by the savings from not having the disaster.

For example, defects corrode trust and trust is the great waste eliminator. Defects are a critical, difficult problem. Defects are addressed in XP by many of the practices: pair programming, continuous integration, sitting together, real customer involvement, and daily deployment, for example. Even if your partner doesn't catch an error, someone else sitting across the room might or it might be caught by the next integration. Some of these practices are certainly redundant, catching some of the same defects.

You can't solve the defect problem with a single practice. It is too complex, with too many facets, and it will never be solved completely. What you hope to achieve is few enough defects to maintain trust both within the team and with the customer.

While redundancy can be wasteful, be careful not to remove redundancy that serves a valid purpose. Having a testing phase after development is complete should be redundant. However, eliminate it only when it is proven redundant in practice by not finding any defects several deployments in a row.

If you're having trouble succeeding, fail. Don't know which of three ways to implement a story? Try it all three ways. Even if they all fail, you'll certainly learn something valuable.

Isn't failure waste? No, not if it imparts knowledge. Knowledge is valuable and sometimes hard to come by. Failure may not be avoidable waste. If you knew the best way to implement the story you'd just implement it that way. Given that you don't already know the best way, what's the cheapest way to find out?

I coached a team that had several good designers, so good that each of them could come up with two or three ways of solving any given problem. They would sit for hours, talking about each of their ideas in turn. By the time they were tired of talking, they could have implemented all the alternatives twice. They didn't want to waste programming time, though, so they wasted talking time instead.

I bought the team a kitchen timer and asked them to limit design discussions to fifteen minutes. When the timer went off, two of them would go implement something. They only used the timer a couple of times, but they kept it around as a reminder to fail instead of talk.

This is not intended to excuse failure when you really knew better. When you don't know what to do though, risking failure can be the shortest, surest road to success.

Sacrificing quality is not effective as a means of control. Quality is not a control variable. Projects don't go faster by accepting lower quality. They don't go slower by demanding higher quality. Pushing quality higher often results in faster delivery; while lowering quality standards often results in later, less predictable delivery.

One of my biggest surprises since the first edition of Extreme Programming Explained was released has been just how far teams have been able to push quality as measured in defects, design quality, and the experience of development. Each increase in quality leads to improvements in other desirable project properties, like productivity and effectiveness, as well. There is no apparent limit to the benefits of quality, only limits in our ability to understand how to achieve higher quality.

Quality isn't a purely economic factor. People need to do work they are proud of. I remember talking to the manager of a mediocre team. He went home on the weekends and made fancy ironwork as a blacksmith. He met his need for quality; he just met it outside of work.

If you can't control projects by controlling quality, how can you control them? Time and cost are most often fixed. XP chooses scope as the primary means of planning, tracking, and steering projects. Since scope is never known precisely in advance, it makes a good lever. The weekly and quarterly cycles provide explicit points for tracking and choosing scope.

A concern for quality is no excuse for inaction. If you don't know a clean way to do a job that has to be done, do it the best way you can. If you know a clean way but it would take too long, do the job as well as you have time for now. Resolve to finish doing it the clean way later. This often occurs during architectural evolution, where you have to live with two architectures solving the same problem while you transition from one to the other. Then the transition itself becomes a demonstration of quality: making a big change efficiently in small, safe steps.

It's always tempting to make big changes in big steps. After all, there's a long way to go and a short time to get there. Momentous change taken all at once is dangerous. It is people who are being asked to change. Change is unsettling. People only change so fast.

I often ask, “What's the least you could do that is recognizably in the right direction?” Baby steps do not justify stasis or glacial change. Under the right conditions, people and teams can take many small steps so rapidly that they appear to be leaping.

Baby steps acknowledge that the overhead of small steps is much less than when a team wastefully recoils from aborted big changes. Baby steps are expressed in practices like test-first programming, which proceeds one test at a time, and continuous integration, which integrates and tests a few hours' worth of changes at a time.

Responsibility cannot be assigned; it can only be accepted. If someone tries to give you responsibility, only you can decide if you are responsible or if you aren't.

The practices reflect accepted responsibility by, for example, suggesting that whoever signs up to do work also estimates it. Similarly, the person responsible for implementing a story is ultimately responsible for the design, implementation, and testing of the story.

With responsibility comes authority. Misalignments distort the team's communication. When a process expert can tell me how to work, but doesn't share in that work or its consequences, authority and responsibility are misaligned. Neither of us is in an intellectual position to see or use the feedback we need to improve. There is also an emotional cost of living with misalignment.

You can use the principles to understand the practices better and to improvise complementary practices when you don't find one that suits your purpose. While the statement of the practices is intended to be clear and objective (for example “write a test before changing code”), understanding how to apply the practice in your context may not be obvious. The principles give you a better idea of what the practice is intended to accomplish. Also, no fixed list of situated, context-dependent practices covers all of software development. You will create new practices occasionally to fill your specific need. Understanding the principles gives you the opportunity to create practices that work in harmony with your existing practices and your overall goals.