The purpose of this appendix is to describe the strengths and weaknesses of Extreme Programming (XP) because XP is the most visible of the agile methods. The message of this appendix should be that XP is good, but that when it comes to achieving sustainable development, XP’s values and practices are a good start but are not sufficient.
Communication.
Clear and effective communication is required from customers (to communicate their needs), developers (to describe what is possible), and managers (to help set expectations) to make an effective project.
A product should only be as complex as is actually required for the problem at hand. Overly complex products have many subtle and unnecessary problems.
Feedback.
Proper feedback stops problems as early as possible in the development process.
Courage.
The team must have the courage to take risks and try new ideas and approaches. Timely feedback will highlight problems as early as possible.
XP’s twelve practices are derived from the four values.
Test Driven Development.
Unit tests written before and during coding allow rapid change by providing confidence that errors aren’t creeping into the product.
Small Releases.
Small releases help ensure that the developers and customers are able to provide each other with continuous feedback.
Refactoring.
Refactoring allows the design to be continually enhanced and simplified over time by reducing redundancy and complexity.
Simple Design.
Reducing complexity is just as important as coding.
Planning Game.
Only plan enough to get started and refine the plan after every release.
Pair Programming.
All production code is written by two people collaborating at the keyboard. This enables collective code ownership and helps to catch coding problems as early as possible.
Onsite Customer.
An onsite customer ensures that feedback is nearly instantaneous so no time is wasted waiting.
A common real-world metaphor that is understood by customers, developers, and managers is used for elements of the project.
Collective Code Ownership.
Collective code ownership helps ensure that no one team member is the critical path or sole repository of knowledge on a particular section of code.
Continuous Integration.
Changes are placed into the shared version control system as early and often as possible so that every developer is working from the latest version of the software. This helps to ensure that problems are caught as early as possible.
Coding Conventions.
Consistent coding conventions aid communication within the team and reduce the amount of time required to learn a new section of code.
Sustainable Pace.
Work at a consistent pace so that consistent and accurate estimates can be made. Use a 40-hour work week to ensure there is also a balance between home and work.
The main strength of XP is that it is designed to appeal directly to software developers. The XP practices, and the emphasis on coaching and self-organized teams are most likely a reaction to software development cultures that stress project management and bureaucracy as a means to achieve successful projects. XP is likely also a reaction to bad managers who tell teams what to do, when, and how to do it.
XP has introduced a healthy debate within the software industry. Traditional engineering methods place decision-making power primarily in the hands of managers through bureaucracy, tracking plans, unnecessary milestones, and documentation as a tracked deliverable. XP counters bureaucracy by placing the decision-making power in the hands of the software development team and the software’s users. Instead, by emphasizing value to customers through working software not documentation, practices that emphasize technical excellence, collaboration between developers, and direct customer contact, XP is steering the industry more toward producing high-quality and high-value results for customers, which is exactly where the emphasis should be.
One of the appeals of XP is that it is extreme, especially compared to traditional software development practices. Pair programming, test-driven development, refactoring, the planning game, etc. are all completely out of the normal realm of experience for traditional software teams. As a result, XP has received a lot of attention and has gotten people to think and question their development methods. The resulting debates have been good for the software industry.
XP’s focus on technical excellence is particularly important. Although it’s hard to identify core practices, a case could be made that they are simple design, refactoring, and test-driven development. Each of these practices is simple yet powerful and helps to emphasize achieving results as quickly as possible so that users can provide feedback and changes made in response to the user comments. Likewise, continuous integration helps teams focus on keeping their software integrated and working through unit tests and the goal of frequent and small releases.
There is no doubt that XP gets teams working together, intimately. Tight collaboration within teams is a vital aspect of success, and by emphasizing pair programming, collective ownership, self-organizing teams, and having an on-site customer focuses teams on collaborating. The result is exhilarating and fun for those involved, and the results are often phenomenal. Greater collaboration helps to ensure that common problems such as duplicated code and effort, unmaintainable code, and independently developed modules unable to integrate with each other. The integration problem is particularly acute; I have seen projects where two developers working in the same cubicle produce modules that can’t be integrated, even when integration was a key design issue!
XP’s collaborative practices help to avoid the hero syndrome. We all know the heroes; they’re the ones who work long hours to get the critical features into the software. At many companies, they’re rewarded by the company for doing so, and there is a general recognition that the company would be in deep trouble if they left. My experience with heroes (and I will confess that in my first job in industry I was one of them) is that they either leave behind a swath of destruction in the code, or they definitely leave behind code that can’t be maintained by anyone else because nobody else knows the code the way they do. While it is true that the best programmers do produce more [Demarco and Lister 1999], often considerably more, there is still plenty of room through collaboration to keep the output of the best programmers high while making the entire team into the heroes through high-value and high-quality results.
The most important aspect of XP’s collaborative practices is the emphasis on collaboration with customers. Because customers are involved in the planning of iterations and prioritization, and because there are regular releases of software to customers, the team is able to get timely feedback that helps to ensure team members are always doing what is most important to the customer. Thus, XP maximizes value to the customer while minimizing the amount of effort required to create that value through decreasing effort wasted on nonessential work.
Many of the weaknesses of XP stem from the fact that it is often presented in a semi-religious, almost cult-like way. This is the view that in order to practice XP, you need to utilize all the practices and stick with them to the letter, with no alternative if you can’t use them or want to add to them. There are some in the XP community who recognize this problem, so hopefully this view will moderate over time.
XP is also subject to problems of interpretation. People who read the XP books or speak to XP teams can easily misinterpret the practices, overzealously apply them, and then decide that since one or two of the practices don’t work, therefore XP doesn’t, when in fact the XP mindset does work, and the XP practices around test-first development, refactoring, and simple design in particular are absolutely brilliant. People who dismiss XP as a whole are missing out on some valuable practices. Unfortunately, you don’t have to look too hard on the Internet to find articles that reflect the failed XP experience.
XP is too narrowly targeted at software developers. Successful software products are developed by teams, and this is stating the obvious, but teams include more than just software developers. Documentation, business people, Quality Assurance (QA), and usability are glaring omissions. Often, these other functions use practices that are incompatible with XP and then struggle with experimental practices, or because there is so much emphasis on the software these other functions are minimized –[md] to the possible peril of the project.
The role of the manager is particularly abused in XP. I know quite a few managers who have read Extreme Programming Explained [Beck 2004] and been turned off on the whole thing because managers are cast in the role of the bad guy. While it can’t be disputed that some managers are the bad guys, I believe that most managers want success and are a part of that success, not a hindrance to it. Perhaps the label manager is the problem. XP disdains bureaucrats (who tell what to do, when, and how) and while it is true that bureaucrats are not desired, leaders who know how to set up a collaborative, empowering environment are definitely required. Some people call this role the coach, and while that does capture the essence of this style of leadership, I believe that coaching is just one attribute of a successful leader.
XP describes a project planning paradigm that is useful for a small team of software developers. Given XP’s focus on software developers, this shouldn’t be a surprise, nor should it be viewed as particularly bad because the practices are useful for some teams. However, the lack of a more rigorous attention on project management has led agile practitioners to propose hybrid agile techniques such as Scrum with XP [Schwaber and Beedle 2001] or ASD with XP [Highsmith 1999] and why Jim Highsmith, for example, wrote his excellent book Agile Project Management [Highsmith 2004a].
XP was developed for the C3 project for Chrysler. It featured a small project team with a single customer. The XP practices alone are hard to scale to larger teams because (among many things) they do not offer any solutions to collaboration between teams, or as mentioned above, do not address any of the issues faced by non-software developers in the team.
Another aspect of scalability is a result of the fact that XP is primarily targeted at software developers. Because XP typically starts in software teams, change is attempted in a purely bottom-up way that can be transferred to other software teams but not through an entire organization without extra practices.
Pair programming is presented as something that software developers absolutely must do. Yet, pair programming is hard: Either you are typing or you are watching someone else type, often poorly or slowly. The reader of the XP books is left with the impression that developers should pair program all day, every day. While pair programming can be very useful and productive, its use should be limited to what the people on the team are comfortable with. It certainly shouldn’t be mandated.
What gets lost in the pair programming debate is that the intent of pair programming is to encourage collaboration between developers and to ensure that problems are caught as the code is typed. Essentially, by forcing pair programming, you force developers to collaborate with each other with the goal of building a team that collaborates by default, and in person not through e-mail or some other contrived means. There are other ways to achieve this same level of collaboration, and it’s much healthier to encourage a range of collaborative practices than to mandate a single one.
The other practice that is often overzealously applied is the 40-hour workweek. I have read articles where XP teams were criticized because, instead of pushing to get a product out the door, they have insisted on working no overtime, and hence lack any sense of urgency. I also have spoken to people who claim to be using XP but work 80-hour weeks on a regular basis, which also isn’t good.
The purpose of the 40-hour workweek is to get teams to work hours that allow them to stay sane and have a life outside work over the long term, measured in many months to years. This sense of balance is important to maintain a sustainable pace of development. The reality is that extra hours are going to be required on occasion, so the emphasis should be on the work/life balance over the mid to long term with a rational average number of hours per week worked. Reasonable working hours and avoiding burnout are a result of the focus on sustainable development, not the focus. Thus, sustainable development requires principles and practices that target sustainability and increasing capability over time.
On-site customers are great if you can get them, or if you can work on your customer site where your customer is readily available. But what if your product has thousands of customers? Tens of thousands? Hundreds of thousands? What if your product sells to customers in many diverse vertical market segments, where each market has unique requirements? The bottom line is that in many cases it is impossible to have a true on-site customer. Many teams who try XP fail because they are unable to get over this hurdle, because XP and all the agile development techniques work best when there is a timely feedback loop between the customer and development team. Unfortunately, this is a rich enough topic for another book, but there are alternatives that are different from having an on-site customer but equally as useful. Just so you aren’t left hanging, here are a few ideas:
Learn about the field of usability or, even better, hire a usability expert or team. Usability professionals have a rich and diverse number of ways to test software with users and get the required feedback that agile teams need.
Use the Internet to ship your software on a regular basis (daily or weekly) and get feedback from your customers so they can provide you with instant feedback and you can solve their problems in an optimal time. More on this in Chapter 4 on Working Software.
Have a customer representative on your team. This person is a user of the product who spends a great deal of his or her time talking to and working with customers on real problems. These representatives funnel the feedback from customers into the team and give team members the feedback they need when they need it.
Many teams get carried away with self-organization and forget that there are often situations where people with specific roles are required. For example, it almost always makes sense to have a project leader to deal with the daily coordination and management of the project. Good project leaders (or coaches) are able to nudge teams forward, while helping them to make the right decisions and know the right decision when they see it. Pure democracy doesn’t always work, and sometimes somebody needs to step forward and make a tough decision where there is no clear consensus, simply in the interest of keeping the project moving forward.
A related topic is that there is a general trend to shy away from the term leadership in XP. As explained in Chapter 8, leadership is a critical success factor in successful projects and is required at all levels of an organization. Leadership is not a top-down thing. It is required at all levels, and effective leaders lead their peers and their bosses, too.
In XP, simple design means only designing what you need to know immediately. As explained in Chapter 6, the problem with this definition is that in many teams this leads to an unhealthy desire to avoid having overall guidelines and design scheme.
Another weakness of XP is that there are some common misconceptions in circulation. I think these are a result of people who read the XP books and try to implement the practices without understanding the principles behind XP.
Unfortunately, there are teams who have adopted XP because they believe it frees them from having to do design. XP is not a license to hack. Just because the emphasis is on working software and simple design does not remove the need to do design (see Chapter 6). No doubt these teams live in an environment where the design burden in terms of undue ceremony and documentation is high, but swinging the pendulum to the complete other extreme is not the correct response either.
Good software developers always think about a problem and plan their approach before they write any code. The difference is that in XP, instead of focusing on a detailed design document as the key output of the design process, the most important attribute is the software and the process you go through to produce the design.
Many people feel that XP teams are not disciplined. Actually, XP requires a great deal of discipline. The misunderstanding comes in through the definition of discipline. Typically, those who think XP teams are not disciplined think of discipline as process compliant, where the process dictates the production of documents in a carefully staged order of analysis then coding. These people use the absence of the documents created in traditional development (e.g., requirements documents, design documents) as evidence that agile teams are undisciplined. There are also feelings of discomfort that agile teams do not put any effort into understanding user requirements or do not do design, when in fact they do. The difference is that with agile approaches, the emphasis is on the software, not documents, and making a decision as late as possible.
Discipline is more than just documents or rules; it is a function of the people in the team. Agile development provides a simple model that can be used to give a project a regular heartbeat with a constant understanding of progress and learning. It is up to the people in the teams to take advantage of the information they have. This is really no different than in traditional development: Discipline is up to the people on the team and not an attribute of the methodology.
Many people have only heard of XP or think that agile development is XP. This is a pity, because there are other agile methods that have a lot to offer. In an environment where a team emphasizes agility and continual improvement in capability through regular retrospectives and experimentation with methods, the other agile methods are an ideal source of ideas and practices. My favorites are:
Adaptive Software Development [Highsmith 1999] and Agile Project Management [Highsmith 2004a]
Lean Development [Poppendieck and Poppendieck 2003]. Lean development is inspired by lean manufacturing [Womack and Jones 1996].
Feature-Driven Development [Palmer and Felsing 2002].
Crystal [Cockburn 2001] [Cockburn 2004].
SCRUM [Schwaber and Beedle 2001].
Dynamic Systems Development Method (DSDM) [Stapleton 2003]. DSDM introduces a healthy business focus to agile techniques.
Agile Development Ecosystems [Highsmith 2002] contains a good summary of agile development and agile methods.
Don’t be deceived by the fact that there is more text in this appendix dedicated to the weaknesses of XP than to its strengths. The strengths more than balance out the weaknesses, and as with any software development method, it is easy to poke holes in the implementation. The practices of XP are an excellent place to start, but XP should not be viewed as a complete methodology.
The most critical element that is missing with XP is the notion of continual learning and improvement in the methods used by the team. Every team is different because it’s made up of different people and faces a different ecosystem. Hence, teams need to place an emphasis on continual improvement of their methods through regular retrospectives and methodology changes that are consistent with the principles (mindset) of sustainable development. This gives them a more healthy balance between software and non-software and should encourage them to continually look for new ideas. If your team is using XP, I highly recommend that you try out new practices as required while staying true to the values. XP unfortunately gets too much attention in the software community. This is both good and bad: good in the sense that XP has caused a large number of developers to question the traditional methods of software development, and bad in the sense that there is more to agile development than XP. There are quite a few other agile methods, and each offers different agile practices or viewpoints that should be consulted and incorporated as a team continually looks for new sources of ideas and inspiration.