C H A P T E R  2

Process Life Cycle Models

If you don't know where you're going, any road will do.

If you don't know where you are, a map won't help.

- Watts Humphrey

Every program has a life cycle. It doesn't matter how large or small the program is, or how many people are working on the project – all programs go through the same steps:

1. Conception
2. Requirements gathering/exploration/modeling
3. Design
4. Coding and debugging
5. Testing
6. Release
7. Maintenance/software evolution
8. Retirement

One's program may compress some of these steps, or combine two or more steps into a single piece of work, but all programs go through all steps.

Although every program has a life cycle, there are many different process variations that encompass these steps. Every life cycle model, however, is a variation on two fundamental types. In the first type, the project team will generally do a complete life cycle – at least steps 2 through 7 – before they go back and start on the next version of the product. In the second type, which is more prevalent these days, the project team will generally do a partial life cycle – usually steps 3 through 5 – and iterate through those steps several times before proceeding to the release step.

These days the management of software development projects generally fall into two different types, traditional plan-driven models,¹ and the newer agile development models.² In the plan-driven models, the process tends to be stricter in terms of process steps and when releases happen. Plan-driven models have more clearly defined phases, and more requirements for sign-off on completion of a phase before moving on to the next phase. Plan-driven models require more documentation of each phase and verification of completion of each work product. These tend to work well for government contracts for new software with well-defined deliverables. The agile models are inherently incremental, and make the assumption that small, frequent releases produce a more robust product than larger, less frequent ones. Phases in agile models tend to blur together more than in plan-driven models, and there tends to be less documentation of work products required, the basic idea being that code is what is being produced and so documentation efforts should focus there. See the Agile Manifesto web page at http://agilemanifesto.org to get a good feel for the agile development model and goals.

We'll take a look at several life cycle models, both plan-driven and agile, and compare them. There is no one best process for developing software. Each project must decide on the model that works best for its particular application and base that decision on the project domain, the size of the project, the experience of the team, and the timeline of the project.

A Model That's not a Model At All: Code and Fix

The first model of software development we'll talk about isn't really a model at all. But it is what most of us do when we're working on small projects by ourselves, or maybe with a single partner. It's the code and fix Model.

The code and fix model, shown in Figure 2-1, is often used in lieu of actual project management. In this model there are no formal requirements, no required documentation, no quality assurance or formal testing, and release is haphazard at best. Don't even think about effort estimates or schedules when using this model.

Code and fix says take a minimal amount of time to understand the problem and then start coding. Compile your code and try it out. If it doesn't work, fix the first problem you see and try it again. Continue this cycle of type-compile-run-fix until the program does what you want with no fatal errors and then ship it.

Every programmer knows this model. We've all used it way more than once, and it actually works in certain circumstances: for quick, disposable tasks. For example, it works well for proof-of-concept programs. There's no maintenance involved and the model works well for small, single-person programs. It is, however, a very dangerous model for any other kind of program.

With no real mention of configuration management, little in the way of testing, no architectural planning, and probably little more than a desk check of the program for a code review this model is good for quick and dirty prototypes and really nothing more. Software created using this model will be small, short on user interface niceties, and idiosyncratic.

_____________

That said, this is a terrific way to do quick and dirty prototypes and short, one-off programs. It's useful to validate architectural decisions and to show a quick version of a user interface design. Use it to understand the larger problem you're working on.

Figure 2-1. The code and fix process (non) model

Cruising over the Waterfall

The first and most traditional of the plan-driven process models is the waterfall model. Shown in Figure 2-2, it was created in 1970 by Winston Royce,³ and addresses all of the standard life cycle phases. It progresses nicely through requirements gathering and analysis, to architectural design, detailed design, coding, debugging, system testing, release, and maintenance. It requires detailed documentation at each stage, along with reviews, archiving of the documents, sign-offs at each process phase, configuration management, and close management of the entire project. It's a model of the plan-driven process.

_____________

Figure 2-2. The waterfall process model

It also doesn't work.

There are two fundamental and related problems with the waterfall model that hamper its acceptance and make it very difficult to implement. First, it generally requires that you finish phase N before you continue on to phase N+1. In the simplest example, this means that you must nail down all your requirements before you start your architectural design, and finish your coding and debugging before you start anything but unit testing, etc. In theory, this is great. You'll have a complete set of requirements, you'll understand exactly what the customer wants, and everything the customer wants, so you can then confidently move on to designing the system.

In practice, this never happens though. I've never worked on a project where all the requirements were nailed down at the beginning of the work. I've never seen a project where big things didn't change somewhere during development. So finishing one phase before the other begins is problematic.

The second problem with the waterfall is that, as stated, it has no provision for backing up. It is fundamentally based on an assembly-line mentality for developing software. The nice little diagram shows no way to go back and rework your design if you find a problem during implementation. This is similar to the first problem above. The implications are that you really have to nail down one phase and review everything in detail before you move on. In practice this is just not – practical. The world doesn't work this way. You never know everything you need to know at exactly the time you need to know it. This is why software is a wicked problem.

All this being said, the waterfall is a terrific theoretical model. It isolates the different phases of the life cycle and forces you to think about what you really do need to know before you move on. It's also a good way to start thinking about very large projects; it gives managers a warm fuzzy because it lets them think they know what's going on (they don't, but that's another story). It's also a good model for inexperienced teams working on a well-defined, new project because it leads them through the life cycle.

So because the waterfall is not a good practical model, it immediately morphs into a slightly different one.

Backing Up the Waterfall

The first thing that happens to the waterfall model is that it changes into the waterfall with feedback, shown in Figure 2-3. This is an admission that a straight-line waterfall doesn't work and that you need the ability to back up to a previous phase when you discover a problem in the current phase.

Figure 2-3. Waterfall with feedback process model

The waterfall with feedback model recognizes that you have to start work with incomplete requirements, design, test plan, and so on. It also explicitly builds in the idea that you will have to go back to previous process steps as new information about your project is uncovered. The new information can be new requirements, updated requirements, design flaws, defects in testing plans, and the like. Any of these will require that you revisit a previous process step to rectify the problem.

This process model is still quite rigid, and it still has the same advantages of a waterfall model when it comes to very large, new projects and inexperienced teams. The two main disadvantages with the waterfall with feedback model are that it really messes with your scheduling big time, and it makes it harder to know when you're finished. It messes with your schedule because in any phase there can be unexpected moves back to a previous phase of development. This also means it's harder to know when you are done.

Because of these disadvantages, the waterfall with feedback model also morphs into a new model, one that attempts to address the scheduling and uncertainty issues.

Loops Are Your Friend

The best practice is to iterate and deliver incrementally, treating each iteration as a closed-end “mini-project,” including complete requirements, design, coding integration, testing, and internal delivery. On the iteration deadline, deliver the (fully-tested, fully-integrated) system thus far to internal stakeholders. Solicit their feedback on that work, and fold that feedback into the plan for the next iteration.

(from “How Agile Projects Succeed”⁴)

While the waterfall with feedback model recognizes that all the requirements aren't typically known in advance, and that mistakes will be made in architectural design and detailed design, it doesn't go far enough in taking those realizations into the process. Iterative process models make this required change in process steps more explicit and create process models that build products a piece at a time.

In most iterative process models, you'll take the known requirements – you'll take a snapshot of the requirements at some time early in the process – and prioritize them, typically based on the customer's ranking of what features are most important to deliver first.

You then pick the highest priority requirements and plan a series of iterations, where each iteration is a complete project. For each iteration, you'll add a set of the next highest priority requirements (including some you may have discovered during the previous iteration) and repeat the project. By doing a complete project with a subset of the requirements every time at the end of each iteration you end up with a complete, working, and robust product, albeit with fewer features than the final product will have.

According to Tom DeMarco, these iterative processes follow one basic rule:

Your project, the whole project, has a binary deliverable. On the scheduled completion day, the project has either delivered a system that is accepted by the user, or it hasn't. Everyone knows the result on that day.

The object of building a project model is to divide the project into component pieces, each of which has this same characteristic: each activity must be defined by a deliverable with objective completion criteria. The deliverables are demonstrably done or not done.” ⁵

So what happens if you estimate wrong? What if you decide to include too many new features in an iteration? What if there are unexpected delays?

______________

Well, if it looks as if you won't make your iteration deadline there are only two realistic alternatives: move the deadline, or remove features. We'll come back to this problem when we talk about estimation and scheduling.

The key to iterative development is “live a balanced life – learn some and think some and draw and paint and sing and dance and play and work every day some,”⁶ or in the software development world, analyze some and design some and code some and test some every day. We'll revisit this idea when we talk about the agile development models.

Evolving the Incremental Model

The traditional way of implementing the incremental model is known as evolutionary prototyping.⁷ In evolutionary prototyping, one prioritizes requirements as they are received and produces a succession of increasingly feature-rich versions of the product. Each version is refined using customer feedback and the results of integration and system testing. This is an excellent model for an environment of changing or ambiguous requirements, or a poorly understood application domain. This is the model that evolved into the modern agile development processes. See Figure 2-4.

Figure 2-4. Evolutionary prototyping process model

______________

Evolutionary prototyping recognizes that it's very hard to plan the full project from the start and that feedback is a critical element of good analysis and design. It's somewhat risky from a scheduling point of view, but when compared to any variation of the waterfall model it has a very good track record. Evolutionary prototyping provides improved progress visibility for both the customer and project management. It also provides good customer and end user input to product requirements and does a good job of prioritizing those requirements.

On the downside, evolutionary prototyping leads to the danger of unrealistic schedules, budget overruns, and overly optimistic progress expectations. These can happen because the limited number of requirements implemented in a prototype can give the impression of real progress for a small amount of work. On the flip side, putting too many requirements in a prototype can result is schedule slippages, because of overly optimistic estimation. This is a tricky balance to maintain. Because the design evolves over time as the requirements change, there is the possibility of a bad design, unless there's the provision of re-designing – something that becomes harder and harder to do as the project progresses and your customer is more heavily invested in a particular version of the product. There is also the possibility of low maintainability, again because the design and code evolve as requirements change. This may lead to lots of re-work, a busted schedule, and increased difficulty in fixing bugs post-release.

Evolutionary prototyping works best with tight, experienced teams who have worked on several projects together. This type of cohesive team is productive and dexterous, able to focus on each iteration and usually producing the coherent, extensible designs that a series of prototypes requires. This model is not generally recommended for inexperienced teams.

Agile Is as Agile Does

Starting in the mid 1990s, a group of process mavens began advocating a new model for software development. As opposed to the heavyweight plan-driven models mentioned above and espoused by groups like the Software Engineering Institute (SEI) at Carnegie Mellon,⁸ this new process model was lightweight. It required less documentation and fewer process controls. It was targeted at small to medium-sized software projects and smaller teams of developers. It was intended to allow these teams of developers to quickly adjust to changing requirements and customer demands, and it proposed to release completed software much more quickly than the plan-driven models. It was, in a word, agile.⁹

Agile development works from the proposition that the goal of any software development project is working code. And because the focus is on working software, then the development team should spend most of their time writing code, not writing documents. This gives these processes the name lightweight.

Lightweight methodologies have several characteristics. They tend to emphasize writing tests before code, frequent product releases, significant customer involvement in development, common code ownership, and refactoring – rewriting code to make it simpler and easier to maintain. Lightweight methodologies also suffer from several myths. The two most pernicious are probably that lightweight processes are only good for very small projects, and that you don't have to have any process discipline in a lightweight project.

The truth is that lightweight methodologies have been successfully used in many small and medium-sized projects – say up to about 500K lines of code. Lightweight methodologies also require process discipline, especially in the beginning of a project when initial requirements and an iteration cycle are created and in the test-driven-development used as the heart of the coding process.

______________

We'll look at two lightweight/agile methodologies, eXtreme Programming, and Scrum.

eXtreme Programming (XP)

eXtreme Programming was created around 1995 by Kent Beck and Ward Cunningham. XP is a “lightweight, efficient, low-risk, flexible, predictable, scientific, and fun way to develop software.”¹⁰

XP Overview

XP relies on the following four fundamental ideas:

• Heavy customer involvement: XP requires that a customer representative be part of the development team and be on site at all times. The customer representative works with the team to create the contents of each iteration of the product, and she creates all the acceptance tests for each interim release.
• Continuous unit testing (also known as test-driven development [TDD]): XP calls for developers to write the unit tests for any new features before any of the code is written. In this way the tests will, of course, initially all fail, but it gives a developer a clear metric for success. When all the unit tests pass, you've finished implementing the feature.
• Pair programming: XP requires that all code be written by pairs of developers. In a nutshell, pair programming requires two programmers – a driver and a navigator – who share a single computer. The driver is actually writing the code while the navigator watches, catching typos, making suggestions, thinking about design and testing, and so on. The pair switches places periodically (every 30 minutes or so, or when one of them thinks he has a better way of implementing a piece of code). Pair programming works on the “two heads are better than one” theory. While a pair of programmers is not quite as productive as two individual programmers when it comes to number of lines of code written per unit of time, their code usually contains fewer defects, and they have a set of unit tests to show that it works. This makes them more productive overall. Pair programming also provides the team an opportunity to re-factor existing code – to re-design it to make it as simple as possible while still meeting the customer's requirements. Pair programming is not exclusive to XP, but XP was the first discipline to use it exclusively.
• Short iteration cycles and frequent releases: XP typically uses release cycles in the range of just a few months and each release is composed of several iterations, each on the order of 4–6 weeks. The combination of frequent releases and an on-site customer representative allows the XP team to get immediate feedback on new features and to uncover design and requirements issues early. XP also requires constant integration and building of the product. Whenever a programming pair finishes a feature and it passes all their unit tests, they immediately integrate and build the entire product. They then use all the unit tests as a regression test suite to make sure the new feature hasn't broken anything already checked in. If it does break something, they fix it immediately. So in an XP project, integrations and builds can happen several times a day. This process gives the team a good feel for where they are in the release cycle every day and gives the customer a completed build on which to run the acceptance tests.

______________

XP Motivation

Risk is the most basic problem in software. Risk manifests itself in many ways: schedule slips, project cancelation, increased defect rates, misunderstanding of the business problem, false feature rich (you've added features the customer really doesn't want or need), and staff turnover. Managing risk is a very difficult and time-consuming management problem. Minimizing and handling risk are the key areas of risk management. XP seeks to minimize risk by controlling the four variables of software development.

The Four Variables

The four variables of software development projects are as follows:

• Cost
• Time
• Features
• Quality

Cost is probably the most constrained; you can't spend your way to quality or schedule, and as a developer you have very limited control over cost. Cost also is where Brooks' law gets invoked (adding programmers to a late project just makes it later).

Time is your delivery schedule and is unfortunately usually imposed on you from the outside. For example, most consumer products (be they hardware or software) will have a delivery date in late summer or early fall in order to hit the holiday buying season. You can't move Christmas. If you are late, the only way to fix your problem is to drop features or lessen quality; neither of which is pretty.

Quality is the number and severity of defects you are willing to release with. You can make short-term gains in delivery schedules by sacrificing quality, but the cost is enormous. It will take more time to fix the next release and your credibility is pretty well shot.

Features (also called scope) is what the product actually does. This is what developers should always focus on. It's the most important of the variables from the customer's perspective and it is also the one you as a developer have the most control over. Controlling scope allows you to provide managers and customers control over quality, time, and cost.

XP recognizes that to minimize risk, developers need to control as many of the variables as possible, but especially they need to control the scope of the project. XP uses the metaphor of “learning to drive.” Learning to drive is not pointing the car in the right direction. It's pointing the car, constantly paying attention and making the constant minor corrections necessary to keep the car on the road. In programming, the only constant is change. If you pay attention and cope with change as it occurs, you can keep the cost of change manageable.

The Four Values

In order for XP to be a viable discipline of development everyone who is involved in an XP project needs to buy into a common set of values that will permeate all the rules that make up the discipline. In XP there are the following four core values that enable it to work:

• Communication
• Simplicity
• Feedback
• Courage

Communication really means spreading the collective knowledge of the group around to all the members. Keeping the XP team small facilitates communication by keeping the number of lines of communication small. Pair programming and collective ownership of the code also facilitate communication by spreading the knowledge of the entire code base around the entire team. XP developers are encouraged to fix bugs they find and to redesign features to make them simpler (see below); this spreads knowledge of the code widely among the team.

Simplicity is key. XP focuses on developing the simplest piece of software that solves today's task. XP developers bet that “...it is better to do a simple thing today and pay a little more tomorrow to change it if it needs it, than to do a more complicated thing today that may never be used anyway.” All developers on an XP team are allowed and encouraged to redesign code to make it simpler at any time. This practice is called “refactoring.” “Concrete feedback about the current state of the system is absolutely priceless. Optimism is an occupational hazard of programming. Feedback is the treatment.”¹¹ XP programmers are required to write tests before they write the code, so that they always have immediate feedback about their code and its impact on the system. Also, the customer is writing functional (acceptance) tests so those are available to measure how well the system is adhering to the “stories” used to develop it.

XP developers must have courage. They must be willing to make changes at any time when the design no longer fits. They need to be prepared to throw code away if it doesn't work. Simplicity supports courage because you're less likely to break a simple system. XP team members track the schedule daily and involve the customer in re-prioritizing features as soon as needed.

The 15 Principles

From the four values described above XP derives some basic principles. The list looks like the following:

Rapid feedback: Get feedback, interpret it, and put it back into the system as quickly as possible. Automated tests are crucial here because you can run unit tests all the time and you can run the entire regression suite whenever you want to integrate your changes.

______________

¹¹Beck, K. (2000)

Assume simplicity: Focus on today's task and solve it in the simplest way possible. This also means that you should be looking for ways to simplify the code whenever you're making changes. Refactoring keeps the code as simple as possible and reduces defects.

Incremental change: Integrate your new code into the system every day. In fact, integrate whenever you finish a task. This allows you to find interface and interaction errors quickly and gives the customer a new baseline to examine at least once a day.

Embracing change: It's gonna happen, so be prepared for it. The whole basis of agile methodologies like XP is that change is a constant in software development and the more your discipline accommodates change, the better your development process will be.

Quality work: Quality isn't free; strive for defect-free code. Pair programming gives you the two heads are better than one gift and test-driven development focuses your code on satisfying requirements. Both of these help lead to fewer defects in your code.

Teach learning: Teach how to learn to do testing, refactoring, and coding better rather than set down a set of rules that say, “you must test this way.”

Small initial investment: The emphasis here is on small teams, particularly at the beginning of a project to manage the resources carefully and conservatively. If you start with fewer resources and a tight budget, it will focus your thinking on lean design and code. This reinforces simplicity.

Play to win: As opposed to playing not to lose. If you don't worry about schedules, or requirements churn, your days will be more relaxed, you'll be able to focus on the problems at hand (and not on the next deadline) and your code will be cleaner, you'll be more relaxed and more productive. Just relax and win.

Concrete experiments: Every abstract decision (requirements or design) should be tested. In XP and in other agile methodologies, you're encouraged to produce something called a spike. A spike is a quick and dirty proof-of-concept piece of code that implements at least the outline of your decision so you can see if you're actually right. Or, if you're wrong, you've not wasted lots of time figuring that out.

Open, honest communication: You have to be able to criticize constructively and be able to deliver bad news as well as good. This is the foundation of a good design or code review. The culture of the team must be that you can offer constructive criticism at any time. The idea is two-fold; first, you're all trying to improve the code, so criticism is a good thing, and second, common code ownership means that everyone is entitled to make changes without fear of hurting someone else's feelings.

Work with people's instincts, not against them: People generally like to win, like working with others, like being part of a team, and especially like seeing their code work. Don't do things that go against this.

Accepted responsibility: The team as a whole is responsible for the product. Responsibility is accepted by the entire team and tasks are not assigned, they're requested. Common code ownership leads to common project ownership. XP teams typically do not have managers that assign work; they have a coach to help with the process and a project manager to take care of the administrative tasks. The development team members themselves select tasks and make sure they get done.

Local adaptation: Change XP to fit your local circumstances and project. This is an application of accepted responsibility. The team owns the project, so the team also owns the process and they reach consensus on adaptations.

Travel light: The team and process artifacts you maintain should be few, simple, and valuable. This implies that you should be willing to change directions quickly and jettison things (code, design) that aren't working for ones that do.

Honest measurement: Measure at the right level of detail and only measure what makes sense for your project. Remember the difference between accuracy and precision.

The Four Basic Activities

In order for XP to take the values and principles just described and create a discipline out of them, we need to describe the activities we'll use as the foundation. XP describes four activities that are the bedrock of the discipline.

• Coding: The code is where the knowledge of the system resides so it's your main activity. The fundamental difference between plan-driven models and agile models is this emphasis on the code. In a plan-driven model, the emphasis is on producing a set of work products that together represent the entire work of the project with code being just one of the work products. In agile methodologies, the code is the sole deliverable and so the emphasis is placed squarely there; in addition, by structuring the code properly and keeping comments up to date, the code becomes documentation for the project.
• Testing: The tests tell you when you are done coding. Test-driven development is crucial to the idea of managing change. XP depends heavily on writing unit tests before writing the code that they test and on using an automated testing framework to run all the unit tests whenever changes are integrated.
• Listening: To your partner and to the customer. In any given software development project there are two types of knowledge. The customer has knowledge of the business application being written and what it is supposed to do. This is the domain knowledge of the project. The developers have knowledge about the target platform, the programming language(s), and the implementation issues. This is the technical knowledge of the project. The customer doesn't know the technical side and the developers don't have the domain knowledge, so listening – on both sides – is a key activity in developing the product.
• Designing: Design while you code. “Designing is creating a structure that organizes the logic in the system. Good design organizes the logic so that a change in one part of the system doesn't always require a change in another part of the system. Good design ensures that every piece of logic in the system has one and only one home. Good design puts the logic near the data it operates on. Good design allows the extension of the system with changes in only one place.”¹²

Implementing XP: The 12 Practices

We (finally) get to the implementation of XP. Here are the rules that every XP team follows during their project. The rules may vary depending on the team and the project, but in order to call yourselves an XP team, you need to do some form of these things. The practices described here draw on everything previously described: the four values, the 15 principles, and the four activities. This is really XP.

• The planning game: Develop the scope of the next release by combining business priorities and technical estimates. The customer and the development team need to decide on the stories (read features) that will be included in the next release, the priority of each story, and when the release needs to be done. The developers are responsible for breaking the stories up into a set of tasks and for estimating the duration of each task. The sum of the durations tells the team what they really think they can get done before the release delivery date. If necessary, stories are moved out of a release if the numbers don't add up. Notice that estimation is the responsibility of the developers and not the customer or the manager. In XP only the developers do estimation.
• Small releases: Put a simple system into production quickly, and then release new versions on a very short cycle. Each release has to make sense from a business perspective, so release size will vary. It is far better to plan releases in durations of a month or two rather than six or twelve. The longer a release is, the harder it is to estimate.
• Metaphor:“A simple shared story of how the whole system works.” The metaphor replaces your architecture. It needs to be a coherent explanation of the system that is decomposable into smaller bits – stories. Stories should always be expressed in the vocabulary of the metaphor and the language of the metaphor should be common to both the customer and the developers.
• Simple design: Keep the design as simple as you can each day. Re-design often to keep it simple. According to Beck, a simple design (1) runs all the unit tests, (2) has no duplicated code, (3) expresses what each story means in the code, and (4) has the fewest number of classes and methods that make sense to implement the stories so far.¹³
• Testing: Programmers constantly write unit tests. Tests must all pass before integration. Beck takes the hard line that “Any program feature without an automated test simply doesn't exist.”¹⁴ Although this works for most acceptance tests and should certainly work for all unit tests, this analogy breaks down in some instances, notably in testing the user interface in a GUI. Even this can be made to work automatically if your test framework can handle the events generated by a GUI interaction. Beyond this, having a good set of written instructions will normally fill the bill.
• Refactoring: Restructure the system “without changing its behavior” to make it simpler – remove redundancy, eliminate unnecessary layers of code, or to add flexibility. The key to refactoring is to identify areas of code that can be made simpler and to do it while you're there. Refactoring is closely related to collective ownership and simple design. Collective ownership gives you permission to change the code and simple design imposes on you the responsibility to make the change when you see it needs to be made.
• Pair programming: All production code written in an XP project must be written by two programmers at one machine. Any code written alone is thrown away. Pair programming is a dynamic process. You may change partners as often as you change tasks to implement. This has the effect of reinforcing collective ownership by spreading the knowledge of the entire system around the entire team. It avoids the “beer truck problem,” where the person who knows everything gets hit by a beer truck and thus sets the project schedule back months.
• Collective ownership: The team owns everything, implying that anyone can change anything at any time. In some places this is known as “ego-less programming.” Programmers need to buy into the idea that anyone can change their code and that collective ownership extends from code to the entire project; it's a team project, not an individual one.
• Continuous integration: Integrate and build every time a task is finished, possibly several times a day (as long as the tests all pass). This helps to isolate problems in the code base; if you're integrating a single task change, then the most likely place to look for a problem is right there.
• 40-hour week: Work a regular 40-hour week. Never work a second week in a row with overtime. The XP philosophy has a lot in common with many of Tom DeMarco's Peopleware arguments. People are less productive if they're working 60 or 70 hours a week than if they are working 40 hours. When you're working excessive amounts of overtime, several things happen. Because you don't have time to do chores and things related to your “life,” you do them during the workday. Constantly being under deadline pressure and never getting a sustained break also means you get tired and then make more mistakes, which somebody then needs to fix. But being in control of the project and working 40-hours a week (give or take a few) leaves you with time for a life, time to relax and recharge, and time to focus on your work during the work-day, making you more productive, not less.
• On-site customer: A customer is part of the team, is on-site, writes and executes functional tests, and helps clarify requirements. The customer's ability to give immediate feedback to changes in the system also increases team confidence that they are building the right system every day.
• Coding standards: The team has ‘em, follows ‘em, and uses 'em to improve communication. Because of collective code ownership the team must have coding standards and everyone must adhere to them. Without a sensible set of coding guidelines, it would take much, much longer to do refactoring and it would decrease the desire of developers to change code. Notice that I said sensible. Your coding standards should make your code easier to read and maintain: they shouldn't constrict creativity.

______________

The XP Life Cycle

The XP life cycle contains all the phases of the generic life cycle described at the start of the chapter, but it compresses the middle three phases – design, code, and test – into a single implementation phase. A productizing phase is added after implementation to allow the code to be stabilized before release. The XP life cycle shows how producing code is the centerpiece of the methodology.

9. Exploration: Exploration is done when “the customer is confident that there is more than enough material on the story cards to make a good first release and the programmers are confident that they can't estimate any better without actually implementing the system.”¹⁵ During exploration, the team's main goal is to get as many requirements (story cards) written as they can. This is also the time when they can explore the architecture possibilities by doing a quick spike of the system. Estimate all tasks done during exploration to practice your estimation skills. In most projects Exploration is the “fuzzy front-end” of the project. You're not quite sure how long it will take and you're gathering requirements and trying to figure out what the product will actually do.

10. Planning game: The Planning game is the tail end of your release exploration phase. In the planning game you need to identify your top priority, high-value stories and agree with the customer which ones will be in the next release. Releases should be from two to six months duration each. Any shorter and you're not likely to get any significant work done and any longer is just plain too hard to plan. Then you need to plan the first few iterations for the release; iterations are 1 to 4 weeks each. Each iteration produces functional test cases for each story scheduled for the iteration. The first iteration helps you nail down your metaphor for the project and puts the architecture in place. Subsequent iterations add new features based on the prioritized list of stories. Reschedule as necessary.

11. Implement: Design, code, test, or actually, design, test, code. One task at a time until all the tasks for a story are complete, and one story at a time until all the stories for this iteration are complete. Need we say more?

______________

12. Productizing: Occurs in the last iteration before your release is done. At this point you should freeze new functionality and focus on stabilizing the product, tuning performance, if necessary, and running acceptance tests.

13. Maintenance/evolution: Well, according to the agile philosophy, you're always in maintenance mode. Here though, you've released something the customer will use and you now must “simultaneously produce new functionality, keep the existing system running, incorporate new people into the team, and bid farewell to members who move on.”¹⁶

14. Death: If the customer can't come up with new stories, mothball the code. If the system can't deliver anymore, mothball the code and start over.

Scrum, mate

The second agile methodology we'll look at is Scrum. Scrum derives its name from rugby, where a scrum is a means of restarting play after a rules infraction. The scrum uses the eight forwards on a rugby team (out of 15 players in the rugby union form of the game) to attempt to (re)gain control of the ball and move it forward towards the opposing goal line. The idea in the agile Scrum methodology is that a small team is unified around a single goal and gets together for sprints of development that move them towards that goal.

Scrum is, in fact, older than XP, with the original process management idea coming from Takeuchi and Nonaka's 1986 paper, “The New New Product Development Game.”¹⁷ The first use of the term scrum is attributed to DeGrace and Stahl's 1990 book Wicked Problems, Righteous Solutions.¹⁸ Scrum is a variation on the iterative development approach and incorporates many of the features of XP. Scrum is more of a management approach than XP and doesn't define many of the detailed development practices (like pair programming or test-driven development) that XP does, although most scrum projects will use these practices.

Scrum uses teams of no more than 10 developers. Just like other agile methodologies, scrum emphasizes the efficacy of small teams and collective ownership.

Scrum is characterized by the sprint, an iteration of between one and four weeks. Sprints are time-boxed in that they are of a fixed duration and the output of a sprint is what work the team can accomplish during the sprint. The delivery date for the sprint does not move out. This means that sometimes a sprint can finish early, and sometimes a sprint will finish with less functionality than was proposed. A sprint always delivers a usable product.

Scrum requirements are encapsulated in two backlogs. The product backlog is the prioritized list of all the requirements for the project; it is created by the scrum team and the product owner. The sprint backlog is the prioritized list of requirements (say user stories) for the current sprint. Once the sprint starts, only the development team may add items to the sprint backlog – these are usually bugs found during testing. No outside entity may add items to the sprint backlog, only to the product backlog.

______________

Scrum projects are facilitated by a ScrumMaster whose job it is to manage the backlogs, run the daily Scrum meetings, and to protect the team from outside influences during the sprint. The scrum master is usually not a developer.

Scrum projects have a daily scrum meeting, which is a stand-up meeting of 15–30 minutes duration where the entire team discusses sprint progress. The daily Scrum meeting allows the team to share information and track sprint progress. By having daily Scrum meetings, any slip in the schedule or any problems in implementation are immediately obvious and can then be addressed by the team at once. “The Scrum master ensures that everyone makes progress, records the decisions made at the meeting and tracks action items, and keeps the Scrum meetings short and focused.”¹⁹

At the Scrum meeting, each team member answers the following three questions in turn:

1. What tasks have you finished since the last Scrum meeting?
2. Is anything getting in the way of your finishing your tasks?
3. What tasks are you planning to do between now and the next Scrum meeting?

Discussions other than responses to these three questions are deferred to other meetings. This meeting type has several effects. It allows the entire team to visualize progress towards the sprint and project completion every day. It reinforces team spirit by sharing progress – everyone can feel good about tasks completed. And finally, it verbalizes problems – which can then be solved by the entire team.

The development team itself is self-organizing; the members of the Scrum team decide among themselves who will work on what user stories and tasks, assume collective ownership of the project, and decide on the development process they'll use during the sprint. This organization is reinforced every day at the Scrum meeting.

Before the first sprint starts, Scrum has an initial planning phase that creates the list of the initial requirements, decides on an architecture for implementing the requirements, divides the user stories into prioritized groups for the sprints, and breaks the first set of user stories into tasks to be estimated and assigned. They stop when their estimates occupy all the time allowed for the sprint. Tasks in a sprint should not be longer than one day of effort.

After each sprint, another planning meeting is held where the Scrum master and the team re-prioritize the product backlog and create a backlog for the next sprint. With most Scrum teams, estimates of tasks become better as the project progresses primarily because the team now has data on how they have done estimating on previous sprints. This effect in Scrum is called “acceleration;” the productivity of the team can actually increase during the project as they gel as a team and get better at estimating tasks. This planning meeting is also where the organization can decide whether the project is finished, or whether to finish the project at all.

After the last scheduled sprint, a final sprint is done to bring closure to the project. This sprint implements no new functionality, but prepares the final deliverable for product release. It fixes any existing bugs, finishes documentation, and generally productizes the code. Any requirements left in the product backlog are transferred to the next release A Scrum retrospective is held before the next sprint begins to ponder the previous sprint and see if there are any process improvements that can be made. Scrum is a project management methodology and is typically silent on development processes. Despite this, Scrum teams typically use many of the practices described above in the XP practices section. Common code ownership, pair programming, small releases, simple design, test-driven development, continuous integration and coding standards are all common practices in Scrum projects.

______________

Conclusion

As can be seen from the methodologies described in this chapter, iteration is the key, whether you are using an evolutionary plan-driven process or an agile development one. Recognize that the best way to build a complex piece of software is incrementally. Learn that designing, writing, testing, and delivering incrementally better code is your first step to writing great software.

References

Beck, K. Extreme Programming Explained: Embrace Change. (Boston, MA: Addison-Wesley, 2000.)

Cockburn, A. Agile Software Development. (Boston, MA: Addison-Wesley, 2002.)

DeGrace, P. and L. H. Stahl. Wicked Problems, Righteous Solutions: A Catalogue of Modern Software Engineering Paradigms. (Englewood Cliffs, NJ: Yourdon Press, 1990.)

DeMarco, T. Controlling Software Projects: Management, Measurement and Estimation. (Upper Saddle River, NJ: Yourdon Press, 1983.)

Martin, R. C. Agile Software Development, Principles, Patterns, and Practices. (Upper Saddle River, NJ: Prentice Hall, 2003.)

McConnell, S. Rapid Development: Taming Wild Software Schedules. (Redmond, WA: Microsoft Press, 1996.)

Paulk, M. C. The Capability Maturity Model: Guidelines for Improving the Software Process. (Reading, MA: Addison-Wesley, 1995.)

Rising, L. and N. S. Janoff. “The Scrum Software Development Process for Small Teams.” IEEE Software 17(4): 26-32. (2000)

Royce, W. W. Managing the Development of Large Software Systems. Proceedings of IEEE WESCON, IEEE Press. (1970)

Takeuchi, H. and I. Nonaka. “The New New Product Development Game.” Harvard Business Review 64(1): 137-146. (1986)