The Project Dilemma
Few projects proceed totally as planned. Most start out with much enthusiasm but often end in a mad rush to get something out the door. The deliverable is often hobbled with inaccuracies, or worse, it invokes unrepeatable responses from the project sponsors. Figure 1-1 shows a time line/effort comparison of person-hour expenditures for most projects that don't meet their commitments.
Figure 1-1. Time line of a typical project
You know the drill: The project plants a stake in the ground, with a big-bang deliverable two and half years away. And then you plod along, making adjustments and adding functionality until, toward the end, you realize, “We aren't going to make it.” You are so far off the mark that you start adding more pounds of flesh to the effort. Before long, you begin to jettison functionality—for example, reporting, archiving, security, and auditing activities. You end up with a poor deliverable that adds to the black eye of the Information Technology (IT) department and further convinces the project sponsors that you can't develop software at all, let alone on time and within budget.
Unfortunately, like lemmings racing to the sea, companies repeat this saga again and again and again.
Iterative and Incremental Software Development
This dilemma of projects failing or not delivering promised functionality stems from the unwillingness of both the IT department and the project sponsors to take a learn-as-you-go approach to software development, more formally known as iterative and incremental software development.
In the context of a software project, many people confuse the terms iterative and incremental. The American Heritage Dictionary, Second College Edition, defines these terms as follows.
Iterative – 3….procedure to produce a desired result by replication of a series of operations that successively better approximates the desired result.
Incremental – 1. An increase in number, size, or extent. 2. Something added or gained….4. One of a series of regular additions or contributions.
Let's give these academic definitions a bit of a software flavor:
Iterative: The application of tasks in a repetitive fashion that works toward bettering an interim or final product.
Incremental: The creation of interim deliverables such that each one adds significant value to the overall project, stands on its own or operates within a well-defined interface, or might take part as a subcomponent of a final product.
As an example, suppose you are constructing a wooden play set for children. You begin by simplifying the project by breaking it up into two incremental parts: (1) tower and attached slide chute and (2) swing and trapeze frame. You realize the project by iterating through the building of each increment. The iterations might be first to create a detailed drawing; then to buy, measure, and cut the wood; next to bolt together the pieces; and finally to stain the wood. Each iteration improves the chances of producing a product that stands on its own. This approach is powerful because many of the same iterative tasks (drawing, sawing, bolting, and staining) are to be applied to each increment (tower/slide and swing/trapeze).
The challenge, however, is to ensure that the first increment will bolt onto (interface with) subsequent increments. You must learn enough about all of the increments that you can approximate how they will work together as an integrated product. Figure 1-2 gives a sample time line for a project using an iterative, incremental approach.
Figure 1-2. Time line of a project with iterative and incremental flow
After years of applying these concepts to many different types of projects, using many different tools and languages, I have no doubt that this is the only way that software can be successfully developed today and in the future.
Risk-Based Software Development
Experience has taught me to always be a silent pessimist when I approach a new project. This idea stems from the repeated observation that something is always lurking nearby that will shift the project's course toward eventual disaster. Although this attitude might seem like a negative way to look at project development, it has saved many projects from disaster.
The project team must always be on the lookout for risks. Risks must be brought to the surface early and often. One way to do this is to extend the project development philosophy so that it is not only iterative and incremental, but also risk based. Appendix A presents project plan templates for the Unified Process. Figure 1-3 shows one possible visual representation of an iterative, incremental project framework founded on a risk-based approach.
Figure 1-3. Iterative, incremental project framework with risk mitigation
One very positive side effect of this approach is the continual improvement of the end product. In addition, risks are addressed promptly because the project components that present the greatest risk are staged first.
The Iterative Software Process Model
It helps to visualize how combining the notions of iterative and incremental development might look graphically. Figure 1-4 shows an iterative and incremental process model. This framework is sometimes called the spiral process model and has been popularized by many practitioners. Each of the four quadrants shown in the figure is labeled as a phase:
Inception
Elaboration
Construction
Transition
Figure 1-4. Iterative and incremental process model: One-dimensional
These four phases parallel the terminology used by the Unified Process from Rational Software. Rational's Unified Process has as its roots the Objectory Process, created by Ivar Jacobson in the 1980s.
The project begins with the Inception phase. This is a discovery phase, a time for solidifying the project vision and incorporating a clear understanding of what features the project will implement. At this point we fly by the entire project, going a mile wide and five inches deep. Out of the Inception phase we will also have determined what the use-cases are. The project will also select the architecturally significant use-cases. It is these requirements that we target in our first iteration in the Elaboration phase. The milestone reached at the end of the Inception phase is called Lifecycle Objective.
In the Elaboration phase, early requirements identified in the Inception phase are solidified, and more rigor is added to the process. The first iteration of the Elaboration phase targets requirements that will have the greatest impact on the project's understanding and development of the architecture. The deliverable from the first iteration in Elaboration is an architectural prototype. This deliverable not only provides the necessary feedback to the user—“Yes, we can build software that coincides with your requirements”—but also it validates the proposed architecture.
Actually, there should be no “hard” architecture decisions to be made after the first iteration in the Elaboration phase. An architectural prototype evolves throughout Elaboration, as more use-cases are tackled in subsequent iterations. The milestone reached at the end of the Elaboration phase is called Lifecycle Architecture. In this book, our target is to make it through the first iteration in Elaboration and deliver an architectural prototype.
The Construction phase includes the mechanical aspects of implementing all the business rules and ensuring that subsystems integrate. This phase is, for the most part, a software manufacturing process. Absolutely no architecture surprises should arise during Construction. It is in the Construction phase that alpha releases of the system are made available to the user. Packaging, rollout, support, and training issues are also dealt with in this phase. The milestone reached at the end of the Construction phase is called Initial Operational Capability.
In the Transition phase, components produced in the Construction phase are packaged into deployable units. At this time in the project the system is typically thought to be in beta status. Some parallel operations may also be taking place alongside existing heritage applications. This phase also details the support issues that surround the application and how the project will be maintained in a production environment. The milestone reached at the end of the Transition phase is called Product Release. In the Unified Process a product release is called an evolution or generation.
Within each phase, multiple iterations typically take place. The number of iterations within each phase might vary (Figure 1-5), depending on the project's unique requirements. For projects with a higher risk factor or more unknown elements, more time will be spent learning and discovering. The resulting knowledge will have to be translated to other deliverables in a layered fashion.
Figure 1-5. Iterations within phases
To make the project even more interesting, the activities traditionally associated with any one phase may be performed in earlier or later phases. For example, if the project has a strong, unique visual component or is breaking new ground, you might need to simulate a prototype during Inception just to generate ideas and solidify the proof of concept going forward.
Combining Iterative with Incremental: Multidimensional View
Given that an iterative process is beneficial, next we envision the previous flow imposed on an incremental delivery schedule. Figure 1-6 illustrates the iterative nature of typical software development projects and shows that different increments will be in different stages of the lifecycle.
Figure 1-6. Iterative and incremental process model: Multidimensional
Notice that each increment (the three upper spirals) is in a different phase of its evolution. Each phase (e.g., Inception) might also be in its own iteration cycle. At first glance, all this might seem overly complex. From the project manager's perspective, more balls do need to be juggled. However, from the perspectives of the user, analyst, designer, and developer, a clear demarcation exists between each increment. The reason for this approach is, once again, to lessen risk by disassembling the logical seams of the application and then attacking each increment individually.