Microsoft’s response to this need follows their line of reasoning behind tools such as Visual Basic, Visual C++, and MFC: Let us (Microsoft) worry about the plumbing, and you (the developer) can worry about solving the business problems. The .NET Framework takes this a step further than any of the offerings before. The .NET Framework provides what is, in effect, a large virtual machine for your software to run on. This framework provides a huge amount of base functionality for you to work with to meet your customer needs. It also enables you to interoperate with existing COM-based technology and write your solutions as components, with different components being written in different languages.

Emerging almost simultaneously with the .NET Framework, with the aim of solving some of the same problems, has been the agile development movement (http://www.agilealliance.org). One of the most well-publicized and written about of the agile development methods is XP. XP aims, through the use of the practices explained in this chapter, to solve these problems of ever-changing needs and creating big, complex solutions.

.NET is a concrete technology (a set of tools and libraries) with which we can work. XP provides us with a set of practices and techniques we can use to better develop our software.

The .NET Framework provides a rich environment for building collaborative applications. Visual Studio .NET provides a rapid development IDE to help developers create these applications. .NET is being marketed as helping developers create solutions that support their business (customers) in a more responsive way; XP practices are centered on creating predictable software for the customer. The common thread between XP and .NET is the central role played by the customer.

Together XP and .NET work well, and most of the development teams I have worked with in the past year have been using both .NET and XP techniques to enhance their offerings to their customers and users. They are what I call eXtreme .NET teams.

If you adopt the whole team practice, the customer is an integral part of the team. Therefore, the customer is always available to discuss issues and resolve them as they arise. This practice reduces the turnaround time for queries and prevents incorrect assumptions from being made. This practice really puts the pressure on the customer role. If a customer really wants software developed, they must put energy into the process. The concept of them being part of the team adds to the reality of one team, one goal.

Never overlook the value of an on-site customer in any project. When both the development team and the customer are discovering new features of a platform together, this practice becomes more valuable than ever before. The immediate feedback that can be provided by the customer enables the team to add really valuable functionality that they may have missed before.

Consider how this applies to our eXtreme .NET team:^[2]

Deepak: Oh did you know we can do this with the .NET Framework? Would that be useful to you?

Chris: Jeez, yes! How long would it take?

Deepak: We think a day or two!

Chris: How certain are you of that?

Eddie: Pretty sure, after all we did a prototype when we were spiking that technology yesterday.

Chris: Wow, well just do it! I’ll take story card^[3] X away until another iteration.

If Chris hadn’t been there to work with the developers, one of several things might have happened:

Any of these (or other) outcomes would have been making assumptions about what the customer actually wanted. Having the customer on site to answer these questions as they come up becomes incredibly valuable.

The planning game practice provides a mechanism for planning out the features in a piece of software. The planning game helps a team estimate timeframes for delivering the software with the features. This practice involves the whole team, including the customer. By including the whole team, the communication between the team members is enhanced. The developers sit with the customer to define the required features and prioritize them. The practice takes the form of a cooperative game. The aim for all participants is to achieve the same end goal: maximize the value of the software produced.

Consider how our eXtreme .NET team interacts during part of their planning game:

Pete: Panic! Time for another planning game.

Chris: Hi guys, I’ve been talking to marketing, and they’ve been doing some research into usage and sales of the application we’re developing. They think that we need to make the application support Tablet PC features.

Sue: No way! Why Tablet PC? We already discussed doing a version for the Pocket PC at a later stage.

Chris: Many more of the users are mobile than we thought. This section of mobile PC users is moving to Tablet PCs. None of our competitors are supporting Tablet features. We can grab that share of the market if we can do this.

Deepak: Okay, a Tablet PC will run our application fine. Tablet PCs run Windows XP professional with extra bits for the tablet.

Chris: Okay that’s good, but what about support for writing in the forms?

Eddie: Which forms? All of them? Where is the most value?

Chris: We think if you can get the onsite report forms and expense claim forms to support handwriting that would be a good start.

Pete: Panic! I have never done anything on the Tablet before; I have no idea how long that will take.

Deepak: Chill Pete, it’s easy. The Tablet SDK just extends the .NET Framework.

Eddie: Yeah, we need to tell Chris how long it will take! Any idea Deepak?

Deepak: I reckon about a day for each form.

Chris: Okay cool, I’ll expect that in the next milestone then.

Eddie: Hold on Chris. You’ve already given us the stories for the next milestone; If you want us to do this Tablet stuff as well, then you’ll have to take something else out!

Chris: Oh yeah! Okay can you swap that with the file-backup feature we discussed last time? You said that was going to take two days.

Eddie: Fine, so we’ll add Digital Ink support to the onsite report and expense claim forms, and we won’t do the backup stuff for the next release.

You can see how this conversation involves all the members of the team. By involving everyone, no knowledge is hidden. Everything is out in the open. Each member of the team knows the current goals.

Pair programming refers to developing code with two developers sitting at one computer. In an XP team, you work with a partner on every piece of production code. Pair programming provides a constant peer review of the code. It helps to spread knowledge throughout the team and trains developers to communicate their intentions more fully as they work together. The aim is to increase the overall intelligence creating each part of the system. The combined intelligence of two developers should be greater than either one of them alone.

As far as information dissemination goes, I have never seen anything work as effectively and as fast as pair programming. One developer learns a new trick on the IDE while messing around after work one evening. The next morning that developer shows her partner what she has learned. The pair then splits when they have finished a task (unit of work, discussed further later on) and work with another two people whom they both show the new trick. Before lunch, four developers have all learned the new technique. By the end of the day, everyone on a team of 12 could easily have learned this new trick. Now imagine this working when each of those 12 developers is learning new things. If they all learn one new thing each evening and then spread the word the next day, the entire team is learning 12 new things a day. In reality, most developers learn more than one new thing a day when they are developing software.

This works best when you have a very dynamic environment with rapidly rotating pairs. I recommend that everyone attempt to swap pairs at least twice a day. The more the better. Some of the best eXtreme .NET teams I have worked with swap pairs between four and six times a day; this is where the extreme comes in!

Test-driven development (TDD) is the practice of writing tests before writing the production code. These tests are a cornerstone that supports many of the other practices, such as refactoring and simple design. The tests communicate the intention of the code and validate that intention each time they are run. This validation helps to increase confidence in the software developed.

TDD is a fantastic example of developers providing their own safety net before walking the tightrope of a new development language or framework. Consider how our eXtreme .NET team has benefited from TDD:

Eddie: So we thought the framework provided this function to perform X.

Sue: Yes, so we wrote a few unit tests and ran them.

Eddie: And, of course, they failed because we hadn’t put any code in yet.

Sue: Yeah, Eddie insists we run the tests before we fill in the code; I hate it, but I know he’s right.

Eddie: So then we created a class A and called method B on it.

Sue: And one of the tests still failed!

Eddie: We couldn’t work it out because that function in Java does just what we wanted, but in the .NET framework it appears to behave slightly differently.

Sue: Good thing we wrote all the tests; otherwise, we could have made a wrong assumption!

In this example, Eddie and Sue were pretty sure the class library would behave as they expected it to, based on their experiences with a different class library. The fact they wrote tests meant they could move forward quickly, fix the problems early, and learn how the class library worked.

The team had another conversation:

Sue: So the next thing we needed was to complete the X task.

Pete: Yes, Sue made us write all the tests first.

Sue: Well, Eddie always does that, and he’s pretty smart, so I thought I’d copy him!

Pete: Anyway, we ran the tests, they failed, and we started coding to get them to pass.

Sue: After writing the code to make only two of the tests pass, we noticed that actually four tests had passed; somehow we had got two tests to pass without doing anything!

Pete: It turned out that the .NET class we were using already does those things for us and so saved us a load of coding.

Sue: So it was worth writing all the tests first. Thanks, Eddie!

Here Sue and Pete discovered through writing the tests some functionality that the .NET Framework classes provided. This meant they could move ahead more quickly without writing code that duplicated the functionality already provided.

The combination of TDD and pair programming meant that two members of the team learned the new feature in the .NET Framework. XP teaches that the more of the practices you use, the greater the benefit you get. The practices play off each other, and the last conversation was a good example of that in action.

Refactoring refers to the process of changing the structure of the code without changing the behavior. Refactoring is a practice that aids the removal of duplicate code and simplifies the code base. Enhancements can be made more easily to refactored code. The practice of constant refactoring means that after each task is complete, the pair reviews their code to see how they might simplify the code without changing the behavior. This practice enables programmers to treat the code in a more dynamic manner. Developers describe their code as becoming more fluid after they learn to refactor effectively.

Refactoring allows an overall architecture to emerge in the system as it is being developed. This approach contrasts with the normal approach of designing the architecture up front and then developing the code to fit with the architecture.

I believe that this is a key component of XP that has really helped many of the teams I have worked with to develop new software with the .NET Framework. Instead of bringing preconceived ideas of how software should be built and trying to make the new .NET code fit into this model, the XP teams have let their architectures emerge. These teams have been able to gain advantages that they would never have gotten if they had attempted to squeeze their code into an old model. This is especially the case for developers coming from either the VB or ASP background.

Let’s join Sue and Eddie as they have just finished a task to see how they use refactoring to change the design:

Sue: Cool, we’ve finished that. I love the way the places all update on the map and in the drop-down list now. Let’s check it in so that we can get the next task finished today.

Eddie: Hey, hold on. We have a code-duplication issue.

Sue: What?

Eddie: The code we copied from the map object into the dropdown list.

Sue: So?

Eddie: We should only have the code in one place; then if we need to change it, we only have to change it in one place.

Sue: Oh.

Eddie: We can refactor the duplicate code into a separate object that both the UI controls use.

Sue: But then the code won’t be in the UI controls where it’s used.

Eddie: Even better! It will be easier to test, and other non-GUI classes can use it.

Sue: I bet there is a design pattern^[4] for what we are doing.

Eddie: You’re right. Will you look it up?

Sue: Sure.

Teams I have worked with who have previously developed large-scale enterprise applications in Java and C++ have been both surprised and pleased with the architectures they have developed. They have told me that if they had done a big architecture design up front, they would never have built the system that way, and it would have been far more complicated than was needed.

XP promotes a form of experimentation and research known as spiking or “doing a spike.” Spiking is experimentation to validate a theory. When the development team needs to know something, a thin vertical slice can be “driven” into the technology to get some answers. These answers can help the project succeed. Most spiking is done without the safety harness of TDD. Spiking is often carried out by solo developers who believe they need a break from pairing. Spikes are also “driven” by the one left out of a team when there are an odd number of developers, so not everyone can pair.

Driving a spike into the area of functionality the development team is planning to work on next will often produce some valuable information. In .NET projects I have worked on I have seen spikes effectively expose new information in the following areas:

All of these examples were areas in which the development team had uncertainties. In a traditional development approach, someone would have been given the task of dealing with that particular issue. That one person would have become the specialist in that area. For some developers, this is fine because they believe they are securing their jobs in the future by being the top gun in that area. This is a false belief. They are in fact acting as a bottleneck to the team’s overall knowledge attainment. By driving a spike and then reporting the result back to the team, the knowledge gets spread, and more perspective can be given to a task. Pairing with another developer to write the tests and production code after a spike is done also helps to spread the knowledge.

Here’s an excerpt from a meeting with our eXtreme .NET team:

Pete: I spiked the transactions in ADO.NET yesterday, and they’ll do the job fine, so I’ll pair with someone to write the code for task X this morning.

Deepak: Did you think about how it works with our project? I’m worried about the threading and that some issues might arise from that.

Pete: Panic! Oh no, I didn’t think of that.

Deepak: Okay, I’ll pair with you this morning, and you can show me what you discovered while we spike how it works across threads.

Pete: Okay, cool.

Here this team discovered that the spike had taken them some of the way to knowing the answer, but more work still had to be done.

After each task is complete, the code written gets integrated with the entire solution. The entire code base is compiled, and all the tests are run to ensure the code just created does not break anything. This continuous integration practice enables the team to ship early and often and enables them to deliver the short iterations on a regular basis.

As part of this practice, automated build and test scripts are developed. These scripts compile and test the entire solution. The scripts are set up as part of the development environment to make the entire process run more smoothly. This practice eliminates the element of human error from the process.

The practices of a methodology are aimed at reducing the chances of something going wrong. In the past, many projects would design and develop modules or components of the system independently. Near the end of the project, the team would attempt to integrate these modules to produce the final solution. This has become known as the big bang approach and is often a rather painful experience—an experience that involves lots of swearing and cursing because the components do not work together as expected. The XP practice of continuous integration aims to prevent this problem from arising by insisting that developers integrate their code after each task is complete and run the entire suite of tests to ensure that their code has not broken anything.

This practice is valuable when developing systems that consist of distributed components, such as Web services, remoting, custom components, or class libraries. Solutions developed in .NET often include many of these components. Even a simple ASP.NET Web site is likely to include some user controls (ASCX files). By carrying out the continuous integrations, the developers get immediate feedback on the changes they have made. The team can then work to fix any issues that arise. When working with a new toolkit, developers are much less likely to fully understand all the implications of the changes they are making. This XP practice helps these developers to cope with this issue in a far more manageable way.

The practice of Stand-Up Meetings occurs each morning. The team stands up and discusses the progress from the previous day and current work issues. Standing up keeps the meetings short and focused.

Learning is about listening and doing. The proof that you have learned something is shown by your ability to teach it to someone else. Stand-up meetings provide an opportunity to spread knowledge to the team. Each morning the team stands up and each member makes a brief statement about his or her achievements the previous day and where he or she wants to get today. This is a great forum for telling the team about some new technology or a new tool that you discovered the previous day. This discussion can then elicit ideas from other team members, as in the following example from our eXtreme .NET team:

Deepak: Yesterday evening after you all went home, I was waiting for my girlfriend...

Whole Group: Ah, how sweet.

Deepak (blushes): Yeah, yeah, anyway I was playing with putting together a simple custom Web control and I discovered this thing where you can use session states; I thought you should all know.

Sue: Wow, that could be really useful for us for the online task list system; can you pair with me this morning and show me how?

Deepak: Sure.

Eddie: If we store those session states in the SQL database, can we then somehow access the task lists from the mobile interface?

Chris: Now that would be useful for my clients! If it can be done, I think you’ll see it as a story in the next iteration.

Problems often result because people are not communicating. This is especially true in efforts that are team focused, such as software development. When the team is not openly communicating, the information flow that keeps the team moving forward is damaged.

Bad communication happens for a reason. Communication is not generally seen as an important skill for developers to obtain. In fact, many developers have previous poor experiences of communication. Managers often do not want to hear what is being said. Other developers have their own issues to contend with and do not want to hear about your issues. This experience does not encourage developers to speak out when things need to be said. A few of these experiences may lead to an environment in which communication is at best limited and at worst nonexistent. A junior developer joining such an environment would not know anything was amiss. His lack of experience could lead to the belief that poor communication is how software should be developed.

The proponents of XP (myself included) claim that communication is vital to develop software of value. The information flow created by ongoing communication allows all members of the team to contribute to the final results. This enables a group of individuals to behave as a team with the goal of developing great software.

Many of the XP practices force communication to be prevalent among the development team. Some of these practices such as the planning game, pair programming, and stand-up meetings are obvious mechanisms, but others such as coding standards and refactoring also lead to greater communication, as you will see in later chapters. An XP team uses the role of the coach to police the communication, ensure it is occurring, and encourage the use of practices to overcome any barriers.

“What is the simplest thing that could possibly work?” is a cry often heard in XP teams. This means to do the simplest thing that will enable you to move forward in the right direction.

Simple is Smart

I often equate the value of simplicity with math problems. I remember at school struggling with a few tricky equations, and then slapping myself on the forehead when I found the answer (because it was so simple). It often takes real genius to create a simple solution from a complex problem. Most people create a complex solution from simple problems and think they must be pretty smart because of the complexity they have created. In fact, the aim is to be as close to genius as possible and try to create simple solutions for the most complex problems. On my own, I have a lot of difficulty trying to be a genius at any time, but with a team of smart people, we can get a lot closer to genius on a regular basis.

Don’t plan the future! The future will happen regardless of whether you plan it. By planning the future, you are always avoiding the present. You are avoiding the present now by planning. You are avoiding the present that will be by focusing on where you are in your plan. A common response I get to this is a declaration of my madness. “Of course you have to plan for the future and design the system first; otherwise, how would you ever be able to write the code? Don’t be crazy!” This is so difficult for developers that it is one of the biggest hurdles to overcome when moving into an XP environment.

As you will learn in this book, XP does not promote detailed upfront design of the entire system. The design is done a small part at a time, allowing for greater flexibility in the software being developed. By developing one small piece of functionality at a time, testing first and constantly refactoring, you do not need to design the system; it will design itself! The designs will emerge as an outcome of taking a disciplined approach to using the practices. The team needs to have experience with object-oriented coding and design patterns so that they can see these patterns emerge and work with them. For this reason, tools such as UML are still incredibly valuable; they enable communication of the patterns as they emerge in the system.

The point of keeping things simple is to code for now rather than never. By this I mean that we should code to solve today’s problems for your customers, not problems in the future that might or might not come up. Doing this gives you a far greater chance of solving those problems more rapidly and providing the customer with some value in the short term. If every day you produce something of value for the customer, then after several weeks the software you are building will have a culmination of all that value. This is often a sticking point for developers.

To see how this is possible, you need to understand how the practices enable us to treat the code in a different way than traditional approaches. The unit tests provided by TDD and refactoring techniques enable us to change code more readily. Creating a more fluid code base means we do not need to be concerned if the decisions we make early on are absolutely correct. We can change the design of the entire system more easily than ever before.

This simple code that you (and your team) have written will ship quicker because less can go wrong. Shipping more quickly enables you to get the value to the customer more rapidly and solicit feedback (see next section) more readily.

Simpler code means less to communicate. By keeping the solution as simple as possible, you are aiding the first value we discussed. Having less to communicate means it is easier to communicate and explain what the code is doing and the emerging design.

One final point: Simple does not mean easy. Developers who are starting out often misinterpret this as “Do the easiest thing you can possibly do.” This is incorrect; simple is often a lot harder to achieve than complicated. As you work through this book, you should start to see how by keeping the task you are working on simple you can build more complex solutions. An example presented later in this book is replacing a switch statement with polymorphic behavior. The switch statement is easier to write, but the polymorphism provides a simpler solution.

Feedback on your progress enables you to adjust your actions to better meet the goals of the project. Feedback leads to a greater understanding of the state of the system being developed. This understanding tells you exactly what you have done right and what you need to work on. Knowing what you need to work on is incredibly valuable. The feedback acts as a guide for the next set of tasks. Understanding an issue early often averts a potential crisis later on.

If you ask an XP team how they are coming along, you might get a response such as “don’t ask me, ask the system.” XP teams build feedback systems into their software using tests, profiling tools, and automated builds. The results from these are generally posted in very public places, providing feedback on their progress for all to see.

The granularity of the feedback cycles goes from minutes to months. The following identifies some of the practices that support these different timescales.

Ship it Quick, Ship it Often

Until it ships, you know nothing! I have personally worked on projects with large scopes where the business needs were not clearly understood. The project managers refused to give the customer anything early on in the development life cycle. The managers would say, “It’s not ready, and they won’t understand that these functions are missing.” This was a flaw in the development life cycle. The development team was coding blind. If we had given the software to the customers earlier on in the process, we could have received feedback on our progress. We were not getting feedback from what we had already developed, and this ultimately led to a failure in the project delivering what the customers wanted.

It is worthwhile shipping the simplest system that makes sense. This simple system forces feedback from the customer, and you can use that feedback to improve the software.

Simplicity, communication, and feedback create a self-supporting cycle; feedback leads to more communication, communication leads to understanding what to test, simple systems are easier to test, easier to test means more feedback, and so on, as shown in Figure 1-1.

Figure 1-1. The self-supporting cycle.

To use the practices prescribed by XP and get the full value from doing so takes courage. Many of the practices are difficult, and if taken at face value, they don’t seem to make sense. It is this courage to take up the practices and make the results happen that contributes to the extreme in XP.

By employing the safety harnesses provided through the use of the practices, developers can increase the intensity of their work. They can go fast and furious at cutting code that solves the problems at hand. By doing this, developers will find they get more produced but may tire more easily; this is called upping the burn rate and lowering the hours. It is about working hard and fast while at work and then getting out to relax and enjoy life. All of these things take courage to do.

Other aspects that require courage include making changes when needed, not before, and being prepared to throw code away. Both of these practices are alien to many developers and might require a change in the way you think about the code you are writing. As mentioned previously, one of the themes of XP is to lower the cost of change. Writing code that we might need in the future actually increases the cost of change. Code you do not need now is code that we are not using but that needs to be changed. The cost of change can be lowered by not including code until it used. The tests help us to validate this; if we find code that is not being used, we should throw it away because it is encumbering the system needlessly.

Employing just courage alone without the safety nets and harnesses provided by the other values and through following the practices is dangerous, often lethal, to the project. Take care and allow your confidence to grow as you follow the practices so that you can have the courage to communicate new ideas. To ensure these new ideas, keep the system simple. Experiment with new ideas and get feedback on new ideas. These ideas will become the code in your software.

One of the least-discussed and yet most important values of XP is respect. This is mainly applied on a human level to the people we work and live with. It is through respect that we can learn how other people think. Knowing how people think enables us to adjust our own behaviors to better work together to accomplish the goals for which we are aiming. Let’s start with our own team of developers. The respect for other developers in our team and other teams around us is vital to the health of our environment. Showing humility for other developers will enable us to learn from them.

Although this book is not about building teams, I recommend that every developer who works in a team of more than one person find out more about team-based behaviors. If every developer did this, the world’s software teams would be much more productive.

I believe every member of a team has some value to offer that team, and it often takes work to instill that belief in other members of the team. Showing disrespect for a member of the team is akin to what is described by Jim McCarthy^[6] as “flipping the bozo bit”; each team member has a bozo bit. Initially, none of us is a bozo, and the bits are all off. When you flip the bit on, that person becomes a bozo and has to work hard to switch the bit off. It is important to respect our fellow team members and not “flip the bit” on them.

Working Together

When I work with development teams, I often hear developers make comments along the lines of “the customer is an idiot; he doesn’t know what he wants, so we’re developing this system that we know he really wants.” It astounds me that such arrogance can really exist in a business environment and people get paid for it (often large sums of money!). Respect for the customer is vital to a successful project. The customer is the person with authority to make decisions, such as whether the team gets paid for doing this work. The problem often stems from the belief that because the customer “can’t count in binary on his fingers,” the customer is “a total loser.” It is true that many customers do not have technical skills, but that is not their job. They are part of the team to bring an understanding of how to create business value from the software being developed. This is a communications issue. It is the job of the development team to work with the customer, in partnership, to produce what the customer does want and ensure that it adds value to the customer’s business.

The management in an organization has a job to do. It often does not include writing code, but this does not make them any less valuable to the project. The management is there to help make your life easier so you can accomplish the job at hand. Have respect for the managers. They are often under pressure to perform and deliver. If you have respect for this, you can work together as a team to accomplish far more than is possible by working against each other.

Finally, it is imperative to respect the users of your software. In one survey done on a piece of software that had been released, 500 users were asked to carry out tasks with the software. The results were disappointing; more than 90 percent of the users failed to carry out all the tasks. The developers, when told this, responded with, “Where did you find so many stupid users?” This lack of respect for the users led to the creation of a piece of software that had all the functionality but was incredibly hard to use; this was not valuable for their business or the customer’s business and the software had to be changed to make it more user friendly. It is important to respect the users and understand how they will use the software.