OBJECTIVES
83
CHAPTER
5
New and Emerging
Process Methodologies
n
Understand the limitations of traditional process methodologies and the
applicability of Agile processes.
n
Understand the basic tenets of Agile software processes.
n
Gain familiarity with several commonly used Agile processes.
91998_CH05_Tsui.indd 83 1/10/13 12:13:16 PM
5.1 What Are Agile Processes?
Agile processes are a family of software development methodologies that produce soft-
ware in short iterations and allow for greater changes in design. It should be noted at
the outset that not all the Agile process characteristics are new and revolutionary. Many
are derived from years of experiences and are similar to the iterative and incremental
process discussed in Chapter 4. Because software development is still very labor inten-
sive, the Agile methodologies also focus upon the human and team aspects of software
development.
A strong consensus among many proponents of Agile methodologies has been
formed around the Agile Manifesto, presented in Figure 5.1. Additional information can
be found at http://www.agilemanifesto.org.
Although there is no absolute definition about what constitutes an Agile method,
there are several characteristics shared by most Agile methods. Unfortunately, a process
is sometimes labeled Agile simply because the author states it to be so. The following is
a list of characteristics and methods that depict Agile methods:
n
Short releases and iterations: Divide the work into small pieces. Release the software to
the customer as often as possible.
n
Incremental design: Don’t try to complete the design up front because not enough is
known early about the system anyway. Delay design decisions as much as possible,
and improve the existing design when more knowledge is acquired.
n
User involvement: Rather than trying to produce formal, complete, immutable stan-
dards at the beginning, ask the users involved with the project to provide constant
feedback. This usually leads to a better-suited system.
n
Minimal documentation: Do only the necessary amount of documentation, which is
just a means to an end. The source code is a big part of the actual documentation.
n
Informal communication: Maintain constant communication, but not necessarily
through formal documents. People communicate better informally. This approach
works as long as understanding is achieved.
n
Change: Assume that the requirements and environment will change, and try to find
good ways to deal with this fact.
Figure 5.1 The Agile Manifesto.
We are uncovering better ways of developing software by doing it and helping others
do it. Through this work we have come to value
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left
more.
84 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 84 1/10/13 12:13:16 PM
When following this interesting and flexible approach, it is important to ensure that
no one abuses the methodology, particularly the one associated with documentation.
Clearly, enough documentation must be available if the released software needs to be
maintained by a group different from the original developers.
5.2 Why Agile Processes?
Although traditional software processes have been applied with much success to some
projects, they usually have some or more of the following problems:
n
Lengthy development times: With traditional development methods, project lengths
ranging from one to five years are not uncommon. For many companies, especially
small- or medium-sized ones, this is definitely too long and not appropriate. A large
number of small businesses may exist for only one year or less. In three years, many
more companies will have changed their main focus or products. Even if the project is
successful by traditional standards, it may be too late.
n
Inability to cope with changing requirements: In most fields, the environment changes
very rapidly, forcing the business to adapt and change. Most software development
projects will have to deal with many requirement changes. Traditional software devel-
opment methods do not handle changing requirements well. They assume that the
later a change is made, the more expensive it will be.
n
Assumption that requirements are completely understood before the project begins: This
is an often unstated assumption for many traditional methods. Although for some
projects it is realistic, for many more it is not. Most users are not capable of expressing
their requirements in clear and unambiguous language. In many cases, they are not
even sure of what they want. In spite of the valiant efforts of many systems analysts,
the requirements will be incomplete, and many times incorrect.
n
Too much reliance on heroic developer effort: Unfortunately, too many software projects
depend on extra development effort to finish on time. Unless the team is extremely
motivated, group members cannot sustain these efforts for too long. After a certain
point, productivity declines, leading to even longer hours.
n
Complex methodology: Most traditional methodologies provide detailed specifications
of activities and work products. Such specifications can be daunting. For example,
Rational Unified Process (RUP) contains more than 100 work products and 30 roles.
Understanding the methodology is time consuming, and most practitioners cannot
afford to become methodology experts.
n
Waste/duplication of effort: Much documentation is mandated, in cluding documenta-
tion that may or may not be needed. Much information is maintained in several forms,
and much care is needed to keep it synchronized. For example, detailed design may be
kept as Unified Modeling Language (UML) diagrams and as source code. Changes to
the code may imply a need to also make changes to UML diagrams, which means the
work has to be done twice. There is the risk of the information not being synchronized
(e.g., the UML diagram having different classes than the code). Keeping all the artifacts
synchronized requires a sophisticated configuration management system. Tool sup-
port can minimize this duplication, allowing you to keep and modify the information
once, and generate the different views.
5.2 Why Agile Processes?
85
91998_CH05_Tsui.indd 85 1/10/13 12:13:16 PM
Agile development methods hold the potential promise to develop software in
smaller iterations, guaranteeing there is a finished product at all times, and demanding
only normal effort from their developers. Agile methods are extremely good at dealing
with change, which means requirements do not need to be completely specified from
the beginning. While most of the success stories come from small- to medium-sized
software projects, we believe that many parts of the Agile methodologies and processes
hold high potential for large projects as well.
5.3 Some Process Methodologies
This section will provide a better idea of what Agile process development methodologies
look like. We believe no particular methodology will be applicable to all projects and all
organizations, and thus the decision process needs to take into consideration the char-
acteristics of the project as well as the organizational culture.
We do not endorse any particular methodology and believe there are many more
Agile methodologies deserving study. Because space considerations restricted the num-
ber of methodologies we could focus on here, we chose a representative sample. From
the traditional Agile methodologies we have chosen Extreme Programming (XP), which
is the most popular, and the Crystal family of methodologies. The Unified Process is one
of the most popular and important frameworks in software engineering today, and we
discuss how it can be used within an Agile context. We also introduce the popular Scrum
process. Finally, we briefly discuss open source software development and highlight the
similarities with Agile development. Interested readers can obtain more information on
these and other Agile development methods through the Suggested Readings section at
the end of this chapter.
5.3.1 Extreme Programming (XP)
Extreme Programming, usually abbreviated as XP, is one of the first and better-known
Agile development methodologies. It was first used by Kent Beck for the C3 project at
(previously known) Chrysler Corporation. Using small teams working in the same room
to encourage communication, XP proposes that only the strictly necessary documenta-
tion is created, with the code and unit tests serving as documentation. This section will
use many of the terminologies from XP, some of which may appear a bit awkward at first.
The core values of XP can be summarized as follows:
n
Frequent communication between team members and with the customer.
n
Simplicity in design and code.
n
Feedback at many different levels. Unit tests and continuous integration provide feed-
back to the individual developer, or pair of developers. Also, small iterations provide
customer feedback.
n
Courage to implement hard but necessary decisions. One possible decision is to not
use XP, if it does not seem appropriate for the project.
In addition, XP follows five fundamental principles that embody the core values:
1. Rapid feedback: Use pair programming, unit testing, integration, and short itera-
tions and releases.
86 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 86 1/10/13 12:13:16 PM
2. Simplicity: Try the simplest possible approach. Don’t worry too much about consid-
ering cases that may or may not occur in the future.
3. Incremental change: Don’t try to make big changes; try small changes that add up.
This is applied to design via refactoring, planning, and team composition as well
as the adoption of XP itself. Code refactoring is a form of code modification to
improve the structure of the code.
4. Embracing change: Try to preserve options for the future while actually solving
your most pressing problems. Delay decisions that commit you to a path until the
latest possible moment.
5. Quality work: Try to create as good a product as possible. Do the best work all the
time. This is assumed to be a natural tendency for most programmers, and it is
encouraged by many of the practices.
XP also proposes several less-central principles: (1) ongoing learning; (2) small initial
investment; (3) playing to win; (4) concrete experiments; (5) open, honest communica-
tion; (6) working with people’s instincts, not against them; (7) accepting responsibility;
(8) local adaptation; (9) traveling light; and (10) honest measurement.
Most of the time, the initial introduction to XP is through the 12 key practices of XP
methodology. Each may be expanded into a lengthy discussion, but only one of the prac-
tices—planning—will be discussed in detail later in this section as an illustration. However,
they can be briefly summarized as follows:
1. Planning: Quickly determine the features to be included in the next release, using a
combination of business priorities and technical estimates. We discuss more about
XP planning later in this section.
2. Short releases: Try to get a working system quickly, and then release new versions
in a very short cycle. Typical release times for XP are two to four weeks. After a
release, the customer runs its tests to see whether the new features actually work
and provides immediate feedback to the team. New detailed plans are made for
the next release.
3. Metaphor: Instead of a formal architecture, use a metaphor as a simple common
vision of how the whole system works. It is simple, so everybody understands it
and can use it to guide their design. However, this is easier said than done. Design
styles and metaphors are difficult to come up with.
4. Simple design: Try to keep the design of the system as simple as possible. Eliminate
unnecessary complexity as soon as you discover it. Do not complicate the design
based on things that might be needed in the future, but choose the simplest solu-
tion that works now. The design may be changed in the future, if necessary.
5. Test-driven development: Ensure that testing is done continuously and is auto-
mated as much as possible. Write unit tests for all code. In certain situations,
test-first development is actually performed. Write the tests before you write the
actual code. Keep running the tests all the time. Ask customers to write functional
acceptance tests to verify when the features are finished. Continue to keep these
tests running after they run the first time.
5.3 Some Process Methodologies
87
91998_CH05_Tsui.indd 87 1/10/13 12:13:16 PM
6. Design improvement (refactoring): Practice refactoring, which involves a restructuring
of the system without changing its behavior, aiming to remove duplication, improve
communication, and simplify or add needed flexibility. Because developers have
unit tests, and continuous integration is practiced, there can be relative confidence
that the system behavior has not changed. The design is not completed all at once
in the beginning, but it is modified when needed and improved over time.
7. Pair programming: Ensure that all production code is written with two programmers
working at the same machine or facility. This is the revision step taken to the extreme.
All code is always reviewed by at least one other person. Although pair programming
does not always sound appealing at first (because two people are required to do the
same job), the improvements in quality usually more than compensate for the small
decrease in productivity. Imagine writing a program with another person. It is quite
embarrassing to write “ugly” code, slack off, or avoid the tests, because a partner is
always present. Also, the design and code has to be understandable, or the partner
may not easily follow the design and the code and will complain.
8. Collective ownership: Establish ownership of the code by the whole team. Anyone
can change any piece of code in the system at any time. Because unit test is
emphasized along with continuous integration, you can be relatively confident
that the changes do not destabilize other pieces of the system.
9. Continuous integration: Integrate the system and build it many times a day, every
time a task is completed. This way, the development organization always has a
working system and can detect integration errors immediately. The assumption is
that the system builds are not like the large software projects that may take half a
day to a whole day to complete.
10. Sustainable pace: Work only at a pace you can sustain; 40 hours a week is reason-
able. As a rule, never work overtime 2 weeks in a row. If a developer constantly
needs overtime, then the original estimate is not correct and the plan may need to
be adjusted. Also, keep in mind that programming is a difficult intellectual activity.
If a person constantly works overtime, his or her productivity decreases, resulting
in accomplishing less work in more time. A fatigued mind also creates more mis-
takes.
11. On-site customer: Include a real customer on the team, always available to answer
questions. This allows development to work without a completely prespecified set
of requirements. The chance of project success increases dramatically for those
fortunate projects that have full commitment from consumers.
12. Coding standards: Ask all programmers to write all code following the same set
of rules, created to facilitate communication through the code. Developers are to
work in pairs with other programmers in the team, and team members are going
to be modifying other people’s code. There needs to be a coding standard that is
abided by all. The particular standard is not that important; most choices are about
equally good. The important thing is for everybody to use the same rules.
88 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 88 1/10/13 12:13:16 PM
These practices are not all equally easy to follow. For example, it is very difficult to
have a full-time user or customer committed to be on-site for answering requirements-
related questions. These customers or users usually have their own work responsibilities
and may not always be available. Working no more than 40 hours a week is another dif-
ficult practice to follow without very enlightened management.
We will discuss planning, the first of the 12 practices, in some detail next as an
example.
Planning Planning is clearly important in any process. In XP, programmers plan only
the immediate next iteration in detail, and changes to the plan can be made during the
iteration if necessary. The basic assumption in XP is that there are four variables that can
be adjusted in a project: (1) scope, (2) cost, (3) quality, and (4) time. Often, the easiest one
to adjust is scope, and it is frequently the one that is changed.
During the planning period, many decisions must be made concerning the project.
Trade-offs have to be weighed, and adjustments have to be made. XP methodology
differentiates the decisions that should be made by business and by technical people.
Business people and customers decide on the following:
n
Scope: What needs to be done in order for the system to be useful?
n
Priorities: What are the characteristics of certain features, and how should they be pri-
oritized?
n
Release scope: What needs to be included in each release?
n
Release dates: What are the important dates to release the software or specific compo-
nents of the software?
The technical staff must make decisions on the following:
n
Estimates: How long will each feature take?
n
Consequences: What are the best choices related to technology issues and program-
ming language? Note that here the decisions may be made jointly with business
people, after certain trade-offs are made and the cost consequence is understood.
n
Process: How are the work activities performed? How is the team organized?
In XP, the traditional planning sessions are called the planning game. Within this
process, release planning is a vital activity performed before each software release to
decide what is to be included in that release. There is also an activity called iteration
planning, which is performed within the iteration. There are typically several iterations
per release.
Functionality requirements are defined through stories, which play a similar role
as features or use cases in other methodologies. Stories are typically written on actual
pieces of paper or story cards. XP proponents believe that using physical cards helps in
several ways, although the stories may be put into a computer later. The actual cards can
be passed around and rearranged as needed.
The goal of the planning game is to maximize the value of the software produced.
The players include development personnel and customers, and the artifacts are story
5.3 Some Process Methodologies
89
91998_CH05_Tsui.indd 89 1/10/13 12:13:16 PM
cards. The strategy is to invest as little as possible to get the most valuable functionality
as quickly as possible. There are three phases in the planning game:
1. Exploration: Find out what the system can do with several so-called moves. The
moves include writing a story, which is done by business personnel. Another move
is to estimate a story, which is done by development personnel who come up with
an ideal engineering time estimate. The estimate does not take into account many
items and assumes no interruptions, meetings, vacations, and so on. A third move
is to split a story if development cannot provide an estimate or if business decides
a specific piece is more important than others.
2. Commitment: Business chooses the scope and date of the next release and develop-
ment commits to delivering it. Here the moves include business sorting the stories
by value into three categories: essential, important, and nice-to-have functional
requirements or stories. Development performs the sorting of stories by risk, again
into three categories: those that can be precisely estimated, those that can be esti-
mated reasonably well, and those that cannot be estimated. Development also sets
what is called the velocity, which is the ratio of ideal time to calendar time. Finally,
the business chooses scope, which is the business requirements expressed on story
cards. This is done through setting a release date and choosing only those story
cards that would fit, or by choosing the story cards and recalculating the dates.
3. Steering: The plan is updated in the steering phase, which consists of several moves
or activities. The first move is at the iteration level, in which business chooses an
iteration’s worth of story cards. The completion of the first iteration should result in a
software system that performs some amount of functions. Recovery is another move.
During this phase, if development realizes it has miscalculated the schedule velocity,
it can ask business to reprioritize the requirements of story cards. A new story may be
introduced if business realizes it needs a new story in the middle of a release. A new
story may replace an existing story or stories with equivalent effort. Reestimate is a
move, or an activity, in which development can reestimate all the remaining stories
and set velocity again, if it feels that the plan needs to be modified.
Within each of the iterations, which are part of a release, the development team inter-
nally subdivides the stories into tasks and carries on a similar planning with the tasks.
Planning is divided into the following tasks:
n
Exploration: The moves are to write a task and to split/combine tasks.
n
Commitment: The moves are to accept a task for which a developer accepts responsibil-
ity, to estimate a task and set a load factor in which each developer estimates his or her
productivity compared with the ideal engineering time, and to balance the tasks for
which programmers add up their tasks and make sure they are not overcommitted.
n
Steering: This phase includes the following moves: (1) implement a task, (2) record
progress status where every couple of days programmers are asked about their task
status and remaining workload, (3) recovery where overcommitted programmers may
ask for help, and (4) verify story, which is essentially executing the functional tests for
the story. The steering phase of the iteration contains some moves that are more than
planning. It actually includes performing the tasks and readjusting the plan for the
iteration, if necessary.
90 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 90 1/10/13 12:13:16 PM
The descriptions of XP methodology are certainly more granular at the programming
level. It seems clear that some of the XP practices would take a very cooperative and
mature set of team members and may also present some challenges to project man-
agement. For example, paired team programming poses a problem to many managers
who have to give individual performance appraisals. The weighing of team effort versus
individual accomplishments for an appraisal can be tricky.
5.3.2 The Crystal Family of Methodologies
The Crystal family of methodologies was developed by Alistair Cockburn, who proposed
the idea that one methodology cannot be adequate for all projects, and that the meth-
odology needs to be adapted to the project. He provided guidance as to what to adapt
and what kinds of practices to use depending on the project.
Cockburn classifies projects according to three factors: (1) their size, which is measured
in maximum number of developers; (2) their criticality, which is gauged by the losses that a
malfunction would cause; and (3) their priority, which is measured by the time pressure on
the project. Note that project size is not measured in lines of code or function points here.
Projects with high pressure need methodologies optimized for productivity, whereas other
projects may prefer to optimize for traceability at the expense of productivity. Furthermore,
the following four levels of criticality as related to quality and complexity of projects are
described: (1) life, which are malfunctions that can cause physical harm to a person, or pos-
sibly loss of life; (2) essential money, which are malfunctions that may cause loss of money
essential for the organization’s survival; (3) discretionary money, which are malfunctions
that may cause loss of money but are not essential for the organization’s survival; and (4)
comfort, which are malfunctions that do not cause measurable monetary loss and yet still
decrease comfort and pleasure to the users.
Cockburn defines some basic principles for software methodologies and describes
how to adapt the methodology to the projects. There are three defined methodologies:
Crystal Clear, Crystal Orange, and Crystal Orange Web. Basically the darker the color is,
the heavier the methodology would be. But these do not cover the complete range of
projects, stopping way short of life-critical or large-scale projects.
Crystal Clear is considered adequate for noncritical projects at the discretionary
money level, and for projects that require teams of up to six or eight people. Crystal
Orange is considered adequate for critical, but not life-critical, projects that require
teams of up to 40 people.
In designing methodologies for the Crystal family, the following underlying methods
should be part of the process:
n
Use larger methodologies for larger teams.
n
Use heavier methodologies for more critical projects.
n
Give preference to lighter methodologies, because weight is costly.
n
Give preference to interactive, face-to-face communication rather than formal, written
documentation.
n
Understand that people vary within a team and with time. People tend to be inconsis-
tent. High-discipline processes are harder to adopt and more likely to be abandoned.
n
Assume that people want to be good citizens; they can take initiative and communi-
cate informally. Use these characteristics in your project.
5.3 Some Process Methodologies
91
91998_CH05_Tsui.indd 91 1/10/13 12:13:16 PM
In addition, there are seven properties that expand on these principles and provide
further guidance, of which the first three are considered extremely important:
1. Frequent delivery: Deliver running and tested code to real users as frequently as
possible, at least every few months. Here delivery can mean several things. At the
highest end, the system is released to the full set of users. A middle ground is to
release the software to a limited set of users who will only test the system. The
lowest end is allowing only user viewing or a demonstration of the system. At this
lowest level, the users may also use the software for a small period of time in a
controlled environment.
The functionality is usually delivered in iterations, which are usually time-boxed,
meaning the release date is fixed with the scope features included and changing
if necessary. This concept of iterations and frequent releases is very much like the
concept proposed in XP.
2. Reflective improvement: Before, during, and after the project, stop to think about
the process and about what can be improved. Improvements can be tried, even in
the middle of a project. Although similar ideas are often mentioned, Crystal is one
of the few methodologies to include this explicitly.
Changing or modifying a process in the middle of the project must be per-
formed with care. The project team must be informed and be ready to alter the
process. The positive side is obviously that the current project, rather than future
projects, will immediately benefit from the reflective movement.
3. Close communication: Encourage close communication among team members.
This communication can be informal and, preferably, face-to-face. In Crystal Clear,
this property is extended into osmotic communication, which means that informa-
tion flows into the background hearing of members of the team. Thus they pick up
relevant information without conscious effort, as if by osmosis. Hopefully, this also
means questions are answered almost immediately without having to spend a lot
of effort in seeking the answer. Osmotic communication basically requires that
all team members are located in the same room, although it could work if small
groups are in separate locations but still in very close proximity.
The remaining four principles are not considered to be absolute requirements.
However, in order to achieve even greater benefits, the following properties
should be included.
4. Personal safety: Encourage team members to speak up without fear of reprisal. This
includes voicing dissatisfaction with some practice or admitting one’s ignorance,
mistake, or even inability to complete an assignment. This provides psychological
security for the team members and enables individuals to give honest feedback.
5. Focus: Addresses the issue of minimizing disruptions and being allowed to con-
centrate on the task at hand. This allows team members to know exactly what
the task priorities are and to focus on the high priority ones. Sometimes a few
team members may be unable to focus because they are constantly interrupted
by other team members who want to take advantage of their special expertise
on the methodology, domain, or technology. Usually teams self-adjust; they may
deal with this problem by establishing a cone of silence—a special time that is set
92 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 92 1/10/13 12:13:16 PM
aside to allow such experts to focus on their own tasks. Although this may sound
elementary, it is often the best designer, coder, or debugger who is also the most
approachable person on the team. They are often inundated with other people’s
problems and thus are unable to perform their own work.
6. Easy access to expert users: Make it possible for the team to get rapid feedback from
expert users on the product, design, requirements, and any changes. Crystal Clear
allows for a few days between the question and the answer. Note that there is a
similar property in XP where users are co-located with the developers to provide
easy and quick access to answers.
7. Good technical environment: Establish an environment that includes automated
tests, configuration management, and frequent integration.
The fundamental items that characterize Crystal Clear and Crystal Orange are sum-
marized in Figure 5.2.
Figure 5.2 A comparison of Crystal Clear and Crystal Orange methodologies.
Crystal Clear Crystal Orange
Teams
Roles/
separate
people
Work
products
Maximum
release length
One team, same room.
At least four people, play-
ing the roles of sponsor,
senior designer, program-
mer, and user.
Other roles may be filled
by the same people,
including project manager,
business expert, or require-
ments gatherer.
Nine items, including
schedule, use cases, design
sketches, test cases, and
user manuals.
2 months.
Different teams for system
planning, project monitor-
ing, architecture, technology,
functions, infrastructure, and
external testing.
Fourteen roles played by
different people, including
(besides those of Crystal
Clear), project manager,
sponsor, business expert,
architect, design mentor, tes-
ter, and UI designer.
Thirteen items, including
those in Crystal Clear plus
requirements documents, sta-
tus reports, UI design docu-
ments, and interteam specs.
Work products are developed
until they are understand-
able, precise, and stable
enough for peer review.
From 2 to 4 months, two user
viewings per release.
5.3 Some Process Methodologies
93
91998_CH05_Tsui.indd 93 1/10/13 12:13:16 PM
There are many characteristics that are common to both Crystal methodologies, and
once again, there are multiple similarities with the XP methodology:
n
Progress is tracked by software deliveries or by major decisions rather than by written
documents.
n
There is automated regression testing.
n
There is direct user involvement.
n
There are two user viewings per release.
n
Methodology-tuning workshops are held at the start and middle of each release.
n
Policy standards are mandatory but can be substituted by equivalent techniques.
n
Coding style, templates, user-interface standards, and so on, are left as local standards
to be maintained by the team.
n
Techniques related to individual roles are left to the individual.
Crystal Clear and XP share many characteristics, with the main difference between the
two being discipline. XP requires strict adherence to design and coding standards, pair
programming, refactoring, and 100% running tests. It also relies as little as possible on
written documentation. Crystal Clear is designed to tolerate variations among people,
requiring a much smaller set of rules.
5.3.3 The Unified Process as Agile
Although the Unified Process is a framework that does not specify particular techniques
for all the phases, it is usually considered a heavyweight methodology that often requires
most, if not all, the requirements to be obtained at the beginning, during the inception
phase. It also requires the architecture and the big design to be completely specified up
front, and its popular instantiations, including the Rational Unified Process, mandate too
many work products or artifacts.
On the other hand, it is an iterative and incremental process, which is one of the main
points of Agile methodologies, and it is designed as a framework, allowing adaptation
to local conditions. Also, the framework itself does not require all of the work products.
RUP is the first and better-known instantiation of the Unified Process, requiring all the
artifacts.
An obvious way to make RUP more in line with Agile methodologies is to restrict the
required work products, eliminate or merge some of the roles, and add more customer
involvement through the iterations. RUP can also be modified by adding one more level
of iterations, repeating all the phases for each iteration. Of course, the devil is in the
details. What exactly are we to eliminate from RUP to make it more Agile?
A number of people are actively working on how to combine the best characteristics
of RUP and Agile projects, either by making RUP Agile or by adding RUP characteristics
to Agile processes. IBM Rational has published several whitepapers and even a special
RUP plug-in for XP.
5.3.4 Scrum
Scrum is another Agile development methodology that has demonstrated good results. It
is lighter in prescriptions than XP, so it is usually associated with some of the XP practices.
94 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 94 1/10/13 12:13:16 PM
This methodology was first introduced by Takeuchi and Nonaka (1986) based on its suc-
cessful usage in the manufacturing industries. Many of the terminologies used in Scrum
are adopted from the game of rugby. Ken Schwaber used this methodology and later
coauthored with Mike Beedle a book titled Agile Software Development with Scrum.
The Scrum framework consists of team roles, events, artifacts, and rules. You must use
all of these components, bbut might add other techniques or components, as well.
Scrum is an iterative method, and is based on short iterations called sprints. Sprints
are basic units of development for units of functionalities of the total software product.
They are short (one month or less) in duration and time-boxed (their duration is held
constant, but scope may be adjusted if needed). Ideally, a potentially shippable product
is produced at the end of each sprint.
A Scrum project defines three core roles (and may have other supporting roles) of
parties who are vital to the success of the project:
n
The product owner, who represents the voice of the customer and ensures that the
team delivers value to the business
n
The development team, usually 3 to 10 developers, who actually produce the software
n
A Scrum Master, who keeps the team on track and ensures Scrum is followed, as a
coach but not as a project leader
There are also two additional ancillary roles: the stakeholders and the managers.
Scrum is based on three “pillar” concepts for its success: (1) transparency (making the
process visible), (2) inspection (of artifacts and progress), and (3) adaptation (whenever
a significant deviation is detected, correct it). An important aspect of transparency is to
have a common definition of when an item is completed so developers, the product
owner, and the Scrum Master can agree on whether an item is finished.
Scrum Events The Scrum process may be summarized as a set of sequential activities or
events. Scrum defines four events (the sprint itself is not considered an event):
n
The sprint planning meeting, in which the product owner and the team decide on what
will be implemented during that sprint.
n
The daily Scrum, a short meeting in which team members synchronize, make sure they
are on-track, and ask for help if needed. A basic tenet is to keep these meetings very
short (15 minutes or less). Oftentimes all participants are standing instead of sitting, to
ensure meetings are short.
n
The sprint review, held at the end of the sprint, to inspect its products and adapt the
product backlog if needed. During this review the product owner identifies what has
been done and discusses the product backlog. The development team demonstrates
the work it has done answers questions, and discusses what went well, what problems
they faced, and how they solved them. Finally the entire team collaborates on what to
do next (which is input for the sprint planning for the next sprint).
n
The sprint retrospective is held after the sprint review and on the last day of the sprint
in which the team members discuss the sprint results, learn from them, and use those
lessons as input for the next sprint planning.
5.3 Some Process Methodologies
95
91998_CH05_Tsui.indd 95 1/10/13 12:13:16 PM
Scrum Artifacts In any software project, it is important to define the artifacts produced
and used to manage that project. Scrum defines the following artifacts to be used for
controlling the project:
n
The product backlog, an ordered list of all the remaining requirements or user stories
for a product. The product owner prioritizes and orders the requirements. Everybody
can see what still needs to be done overall (although developers focus mostly on the
sprint backlog).
n
The sprint backlog is the ordered list of tasks need to be done for the current sprint.
Tasks here are broken down to be small (4 to 16 hours usually), so developers know
exactly what to do. Rather than tasks being assigned, developers choose their next
task based on the sprint backlog and their particular skills.
n
The increment is the current project; that is, the sum of all requirements implemented
in this sprint and all previous sprints. This should be a shippable (albeit not feature-
complete) project.
n
The burn down chart, which is a frequently updated (daily or more often), publicly dis-
played chart showing the remaining log in the current sprint backlog.
Scrum is an Agile methodology that has demonstrated good results in many proj-
ects; there are currently many organizations using it, improving and adapting it, and
combining it with other techniques, such as Kanban. Kanban is a lean and just-in-time
methodology initially used in the automotive industry in Japan. Many organizations
are now scaling Scrum to bigger projects, having several teams, and holding “Scrum
of Scrums” meetings.
5.3.5 Open Source Software Development
In the past few years, open source software development has emerged as a successful
model. Such software is made available, usually for free, in executable form, but access to
the source is also provided. In addition to access to the source, permission to redistribute
and to modify the source is granted. This leads to a situation in which many developers
can modify and improve the program.
There are many successful open source programs, including the Apache web server,
the most widely used web server program, the Linux and BSD operating systems, and
Mozilla, a web browser integrated with other related programs. Mozilla includes a
mail program and a page designer that are now also distributed as individual compo-
nents, with Firefox, the popular web browser. Several popular database servers such
as PostgreSQL and MySql, and many Internet infrastructure programs such as Bind,
Sendmail, or Postfix are also open source programs.
There is a tremendous variation in open source programs and their development,
with some of them being essentially developed by one corporation such as the MySql
product. Others, such as Linux, have strong corporate backing, and some are developed
by individuals without any monetary compensation. Other variables include the team
sizes, number of people involved, and the length of time the program has been open
96 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 96 1/10/13 12:13:16 PM
source. Many programs started as commercial ventures and turned into open source
when the business cases presented themselves.
In spite of those differences there are some similarities within the development pro-
cess of many open source programs and many similarities with Agile methods:
n
Small releases: In the open source world, the phrase is “release early, release often.”
Most of the time releases are very frequent, ranging from a few weeks to a few months,
and the source is available on the Internet even between releases.
n
Informal, written communication using Internet tools: Communication is done infor-
mally, mostly through bulletin boards and mailing lists. Although it is not face-to-face
communication, it is not formal documentation. One advantage of this is that com-
munications are easily archived and disseminated.
n
Customer availability: For most successful open source projects, the developers are the
initial customers. Most programs start with somebody’s need to “scratch an itch.” That
is why most successes are in areas of programming tools and systems infrastructure. A
common cause of failure is lack of customer feedback.
n
Continuous integration: Systems are integrated often, usually through Internet tools.
n
Shared vision: Successful open source projects usually have strong leadership that
promotes a shared vision. For example, Linus Torvalds, the creator of Linux and the
closest entity to a maximum authority, has been described as a benevolent dictator
who makes most of the architectural decisions. If this vision is not shared, developers
abandon the project, or sometimes “fork” it, and start a new branch from the original
source code.
Of course, there are many differences between open source projects and most Agile
methodologies:
n
Larger teams: Although the core team for most open source projects is quite small,
ranging from one or two people to a few dozen, it is often larger than most Agile
teams. Also, many more users and developers provide feedback, and add small
changes and features.
n
Distributed teams: In many cases the team members involved are from various coun-
tries or regions, which means that communications are often not synchronous. Most
Agile methodologies put the team in the same room.
n
Scaling: Some successful open source projects have achieved massive scale. The Linux
kernel, for example, contains several million lines of C code. Most Agile projects are of
a much smaller scale.
There are other factors affecting the success of open source projects, including the
programmers’ expertise. Most of these successful projects include exceptionally skilled
and motivated programmers. Many of these programmers work for free, obtaining satis-
faction from intangibles. Surprisingly, this actually increases their motivation.
There is still much to be learned about open source development methods, including
a wide assessment of what would make them successful. We can see that the successful
ones share certain characteristics, but we do not know how many other projects with the
same characteristics have been unsuccessful. It appears that open source development
5.3 Some Process Methodologies
97
91998_CH05_Tsui.indd 97 1/10/13 12:13:16 PM
methods will become more important in the future, especially as many of their tech-
niques become better understood and adapted into more mainstream methodologies.
5.3.6 Summary of Processes
This chapter has focused on several processes, most of them related to Agile methodolo-
gies. Table 5.1 summarizes the processes covered in this chapter.
In the next section we will discuss how to choose a process for a project, and provide
some direction in choosing methodologies and process.
Table 5.2 compares the main characteristics of each kind of process.
5.4 Choosing a Process
We firmly believe that there is no process that can fit all projects. The process needs
to be adjusted to the project, to the organizational culture, and to the people par-
ticipating in it. Both Agile and traditional processes can be used successfully for many
projects.
A problem that frequently occurs when deciding which kind of process to use is that
there are many variations within each class and even within methodologies. Both the
Unified Process (UP) is actually a frameworks that can, and should, be adapted to better
suit the project. The Rational Unified Process, described in Chapter 4, is an instantiation
Table 5.1 Summary of the More Recent Methodologies and Processes
Method Main Points Agility Discipline
XP Most popular Agile process. Requires high discipline and adher-
ence to principles and practices. Based on 4 core values (commu-
nication, simplicity, feedback, courage), 5 fundamental principles
(rapid feedback, simplicity, incremental change, embracing
change, quality work), and 12 practices.
High High
Crystal Clear Extremely light in methodologies. Does not require adherence to
all principles. Based on 7 principles: frequent delivery, reflective
improvement, close communication, personal safety, focus, easy
access to expert users, and a good technical environment. Will
only work for small projects and teams.
High Low
Crystal Orange A heavier methodology than Crystal Clear and suitable for larger
projects. It has different teams for different functions. Still not
suitable for large or life-critical projects.
Medium Medium
Scrum A disciplined methodology that can be adapted and combined
with other techniques. It provides clear visibility to the project
status and, in the long run, reduces the effort spent on project
management.
High High
RUP Framework, usually instantiated as a very heavy process. Can
be pared down to a relatively Agile process. Covered in depth in
Chapter 4.
Low to medium High
98 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 98 1/10/13 12:13:17 PM
of the UP that also recognizes the need for adapting the project. XP, and most of the
Agile methodologies, also recognize this need. Local adaptation is one of the XP tenets.
When a success or failure is reported on projects using XP or other Agile methods, it is
necessary to verify which pieces of the Agile methods were used.
5.4.1 Projects and Environments Better Suited for Each Kind of Process
Table 5.3 compares the kinds of projects and environments better suited for each kind
of methodology, using XP as an example of an Agile process and RUP as an example of a
traditional one. It is assumed here that the methodologies will be used unchanged rather
than in piecemeal fashion.
5.4.2 Main Risks and Disadvantages of Agile Processes
Earlier in this chapter we discussed some of the problems with traditional development
methodologies. We have described several Agile methodologies and how they address
these shortcomings. The following disadvantages are related to Agile processes:
n
Possibly not scalable: Agile processes have been utilized by relatively small teams, and
are not able to scale without losing some, or a lot, of their agility. Many projects are too
big or too critical to be developed with Agile methods; nevertheless, many of the Agile
practices and ideas can be incorporated into traditional methodologies.
Table 5.2 Characteristics of Agile Versus Traditional Processes
Agile Traditional/Heavy
Requirements Assumes they will change; requirements are
collected informally at the beginning of the
project, and then at the beginning of each
iteration. Uses constant user interaction
instead of formal requirements.
Assumes they will not change during the
project. A complete, detailed formal require-
ments document is necessary for success. Any
changes in requirements after the design or
implementation has started will be costly.
Design Informal and iterative. Formal and done up front, after all require-
ments are known.
User involvement Crucial, frequent, throughout the whole
process.
Required only at the beginning (requirements
solicitation and analysis) and at the end
(acceptance testing).
Documentation Minimal, only what is necessary; relies on
source code as the ultimate documentation.
Usually requires heavy, formal documentation
of every phase of the project.
Communication Done informally, throughout the project. Relies mainly on documents and formal
memos and meetings.
Process complexity Relatively low; initial description contains less
than 200 pages.
High. RUP (in 2002) describes more than 100
artifacts, 9 disciplines, 30 roles, and 4 phases.
Overhead Low. Relatively high, although it can be scaled
down for smaller projects.
5.4 Choosing a Process
99
91998_CH05_Tsui.indd 99 1/10/13 12:13:17 PM
n
Heavy reliance on teamwork: Not all people are able to work well in teams. Often, one
bad member can destroy the cohesiveness of the entire team. Agile methods rely on
informal communication and team dynamics much more than traditional ones.
n
Reliance on frequent access to customer: Agile methods need frequent feedback from
the customer, the on-site customer in XP, and the small releases with feedback from
customers. This level of constant customer feedback will simply not work when a
team is set to develop a large enterprise application such as PeopleSoft or SAP. In
these large industry-wide applications, multiple industry experts across a span of 10
to 20 companies are involved in the requirements process. Because these interactions
with the customers are not free, they have to be well planned and coordinated ahead
of time. Not all customers are willing or able to provide this level of cooperation and
feedback. Without customer feedback, Agile methods are not able to validate require-
ments or adapt to change. Traditional methodologies concentrate the feedback at the
beginning of the process during the requirements phase and at the end of the process
during the acceptance tests phase.
n
Cultural clash: Many XP practices clash with accepted software engineering wisdom,
or with common management techniques. Performance evaluations, for example, are
harder to perform in XP because the work is done in pairs, and the code is owned by
the whole team.
5.4.3 Main Advantages of Agile Processes
There are some distinct advantages to adopting Agile processes. The most commonly
mentioned ones are as follows:
n
Low process complexity: The processes themselves are simple, which allows them to be
easily understood and implemented.
Table 5.3 Projects for Agile Versus Traditional Processes
Agile (XP) Traditional (RUP)
Scope/size Small; limited to 1 team of up to 10 people. Better suited to larger projects; scales up to the
largest projects; can be scaled down for smaller
projects.
Criticality Relatively low; not suitable for life-critical
systems without adaptation.
Can be used for mission-critical systems (maybe
with minimal modifications).
People More suitable for team players, “good citi-
zens” who can do design and programming
adequately. XP requires strict adherence to
certain practices.
Defines many roles, which can be appropriate
for most kinds of people; doesn’t require tight
team playing; almost any personality will work,
as long as the team member can follow rules.
Company culture Better suited for small co-located companies
with relaxed cultures.
Better suited for larger companies with possibly
geographically remote sites and more formal
cultures.
Stability Copes easily with changes in requirements or
environment.
Less suited to cope with changes. Assumes a
relatively stable environment where require-
ments don’t change much. Can be adapted.
100 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 100 1/10/13 12:13:17 PM
n
Low cost and overhead: The processes mandate very few activities that do not directly
produce software.
n
Efficient handling of changes: The processes are designed with the assumption that
requirements will change, and the methodology is prepared for these changes.
n
Fast results: Most Agile processes have fast iterations and produce a core system that
can be used in a relatively short time. The system will then be improved and more
functionality added as the project progresses. Given that the processes have low over-
head, they also tend to produce final results faster. It is made for continuous integra-
tion.
n
Usable systems: Because the customer is involved and the process deals well with
changes, chances are the final product will be what the customers actually want when
the project is completed rather than what was originally planned as requirements.
5.5 Summary
In this chapter we have covered some of the principles underlying Agile methodologies
as well as several specific methodologies. We have also compared Agile methods with
more traditional ones, specifically RUP.
We believe that processes and methodologies have to be adjusted for each project and
have presented some of Cockburn’s ideas about how to do so. A software engineer must
know about many different methodologies to be able to adopt specific techniques that
may be useful to his or her particular project.
5.6 Review Questions
1. List the four core values of XP.
2. List five XP practices.
3. What factors does the Crystal family consider when choosing a methodology?
4. Explain some of the characteristics of Agile methodologies.
5. Explain the main aspects that open source development shares with Agile meth-
ods.
6. Compare and contrast Agile and traditional methods.
7. Agile methods prefer working programs over comprehensive documentation.
(True/False)
8. Agile methods prefer rigid processes over adapting to the people. (True/False)
9. What is test-driven programming, and which Agile process advocates it?
5.7 Exercises
1. As a class project or just for experience, try pair programming. Get in touch with a
person you would like to work with, and agree to complete this project only as a
5.7 Exercises
101
91998_CH05_Tsui.indd 101 1/10/13 12:13:17 PM
pair. After doing so, analyze your experience. Was the software better than if you
had written it alone? Did it take more time than if you had done it alone?
2. As a class project or on your own, try test-driven programming. Try to write test
cases before you write the code, and make sure all your cases run all the time.
Chapter 10 provides more information about unit testing and test-driven devel-
opment.
3. Consider a large, industry-specific product such as a hospital management system
for a group of hospitals. Discuss the pros and cons of using Agile methodologies
versus a traditional process where more rigid planning and documentation are
required. Focus on issues such as project size, team size, continuing multiple
releases over the years, expanding the product for an international market, and
worldwide customer support.
5.8 Suggested Readings
P. Abrahamsson, O. Salo, J. Ronkainen, and J. Warsta, “Agile Software Develop-
ment Methods: Review and Analysis,” VTT Publication #478, http://www.vtt.fi/inf/pdf
/publications/2002/P478.pdf, accessed January 2013.
K. Beck, Extreme Programming Explained: Embrace Change (Reading, MA: Addison-Wesley,
1999).
A. Cockburn, Agile Software Development (Reading, MA: Addison- Wesley, 2001).
——“Just in Time Methodology Construction,” http://alistair.cockburn.us/just-in-
time+methodology+construction, accessed September 2012.
L. Mathiassen, O. K. Ngwenyama, and I. Aean, “Managing Change in Software Process
Improvement,” IEEE Software (November/December 2005): 84–91.
G. Pollice, “Using the IBM Rational Unified Process for Small Projects: Expanding Upon Ex-
treme Programming,” http://ftp.software.ibm.com/software/rational/web/whitepapers/
2003/tp.183.pdf, acces sed September 2012.
E. S. Raymond, “The Cathedral and the Bazaar,” http://www.catb.org/~esr/writings/
cathedral-bazaar/, accessed February 2005.
K. Schwaber and M. Beedle, Agile Software Development with Scrum (Prentice Hall, 2002).
H. Takeuchi and I. Nonaka, “The New New Product Development Game,” Harvard Business
Review (January 1986): 137–146.
102 Chapter 5 New and Emerging Process Methodologies
91998_CH05_Tsui.indd 102 1/10/13 12:13:17 PM
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.