The Principles of RUP

The Rational team based their framework on six key principles for business driven development.⁴

• Adapt the process. The project or organization must, as we saw previously, select the parts they need from the RUP framework. Things to consider here are, for example, how project governance, project size, regulations, and such issues affect the degree of formality that should be used. There are preconfigured process templates for small, medium, and large projects in RUP so that we can choose more easily. Most companies that I have seen usually adapt RUP in their own way. One of my former employers had several different RUP adaptations based on different project types.
• Balance stakeholder priorities. RUP tries to take a shot at balancing out the business goals and stakeholder needs between the parties involved, because these often differ and are conflicting.
• Collaborate across teams. Hopefully as we all know, software engineering is a team process. We have various participants in a project, from stakeholders to developers. In Chapter 1 we saw that much development these days does not happen at one location, but could actually be geographically dispersed all over the world. This means that collaboration and communication between participants must be good—not only for requirements issues but also for all aspects of the development process. Project status, test results, bug status, release management, design and architecture diagrams, and much more must be at hand for those who need it, as well as at the time they need it.
• Demonstrate value iteratively. One problem with the waterfall model is that it does not allow us to go back a phase if we find things in one phase that throws things in earlier phases to the ground. By working iteratively we deliver working software in increments. For each iteration we collect feedback from everybody, including stakeholders, and use this as an input to the next iteration. This way we can influence the development process and hence the business value while the project is executed. By focusing strongly on iterative development and good risk management, RUP allows projects an iterative risk assessment process that is intended to ease the effort in delivering a successful project in the end.
• Elevate the level of abstraction. By elevating the abstraction level, RUP encourages the use of software patterns, 4GL, frameworks, reusable components, and so on. This approach hinders developers from going directly from the spec to writing their own custom-made code. This also means that we discuss architecture at a higher level than before. By using UML (Unified Modeling Language) or some built-in features of the development tool (see Chapter 6 for Visual Studio’s architecture tools) in conjunction with a higher abstraction level, we elevate product architecture to a level where nontechnical stakeholders can better participate.
• Focus continuously on quality. Surprisingly enough we do not focus on quality enough during many projects. I have had contractors at the Swedish Road Administration that didn’t have this in focus in their projects. Instead, their primary goal was to suck as much money as possible from the SRA (and from me as a taxpayer). This only caused problems, as you would guess, because if the SRA did not keep an extra eye open, the projects were unsuccessful. RUP encourages continuous quality checks through development. Automation of test scenarios for example helps us deal with an increasing amount of tests caused by the iterative process and the practice of test-driven development.

The attentive reader (yes, I mean you!) has already noticed that if we take the starting letter from each of these principles we get the ABCs of RUP:

• Adapt the process
• Balance stakeholder priorities
• Collaborate across teams
• Demonstrate value iteratively
• Elevate the level of abstraction
• Focus continuously on quality

The RUP Lifecycle

So, what does the RUP lifecycle look like? There are four major phases. And no, we do not talk waterfall here (see Figure 3-3). The phases are

• Inception
• Elaboration
• Construction
• Transition

Inception Phase

As Figure 3-3 shows, the Inception phase has a strong focus on business modeling and requirements specifications. The difference from the Waterfall model is that we do not close these topics after the phase has ended. Instead, they are a constant part of all phases through the project until its end. This phase establishes a baseline so that we can compare actual expenditures to planned expenditures along the project. Before we move on to the next phase we need to pass a milestone called the Lifecycle Objective Milestone (see Figure 3-4).

To pass this milestone we need to meet these criteria:

• Stakeholder concurrence on scope definition and cost/schedule estimates.
• Agreement that the right set of requirements has been captured and that there is a shared understanding of these requirements.
• Agreement that the cost/schedule estimates, priorities, risks, and development process are appropriate.
• All risks have been identified and a mitigation strategy exists for each.

If we are not satisfied with the outcome of this milestone or the phase, we can choose to cancel or report this phase for redesign.

Elaboration Phase

During the Elaboration phase we start to see what the project will look like. In Figure 3-3 you see that analysis and design has its biggest effort here but that it will be required to continue through the other phases. There are also other activities we perform in this phase. We start to think about the implementation, how the code will be written, what to code, and so on. The thought is that most use cases are developed during elaboration, where actors are identified and the flow of the use cases are thought out.

To pass the Lifecycle Architecture Milestone that finishes the Elaboration phase (see Figure 3-5) we should have completed 80 percent of the use case models.

We should also have created a description of the architecture of our software. The risk list should have been written, as well as a development plan for the entire project. These are the main criteria for passing the Lifecycle Architecture Milestone:

• Is the vision of the product stable?
• Is the architecture stable?
• Does the executable demonstration show that the major risk elements have been addressed and credibly resolved?
• Is the plan for the construction phase sufficiently detailed and accurate? Is it backed up with a credible basis of estimates?
• Do all stakeholders agree that the current vision can be achieved if the current plan is executed to develop the complete system, in the context of the current architecture?
• Is the actual resource expenditure versus planned expenditure acceptable?

There are a few more criteria we must meet before we can pass this milestone, but we will not go into them here. If we cannot pass the milestone, we can either cancel or redesign, just like in the preceding phase. When we continue to the next phase, project changes are more difficult to solve if we do not have a model that covers such events.

Construction Phase

Now we are ready for the Construction phase. This is where the coding starts and when we will implement our architecture. To make sure we catch changes of requirements we do the development in iterations, each delivering a working prototype. We can show this to stakeholders and end-users so that they have a chance to provide feedback on it. When going in to this phase the use cases have been prioritized and divided across the iteration. One good practice is to focus on the highest risk use cases first, or at least as early as possible, so that we can catch their implications early. To end this phase, we must pass the Initial Operational Capability Milestone (see Figure 3-6) by answering the following questions:

• Is this product release stable and mature enough to be deployed in the user community?
• Are all stakeholders ready for the transition into the user community?
• Are the actual resource expenditures versus planned expenditures still acceptable?

Transition Phase

When we reach the last phase, the Transition phase, we have moved our system/software from the developers to the end-users. This phase, as well as the Elaboration and Construction phases, can be performed iteratively. During transition we train the end-users and the operations department in their new system. We also do beta testing of the system to make sure we deliver what the end-users and stakeholders expect. This means that we not necessarily have the same expectations as when the project started but expectations that have changed through the process. If we do not meet either the end-users’ expectations or the quality level determined during Inception, we do a new iteration of this phase. When we have met all objectives, the Product Release Milestone (see Figure 3-7) is reached and the development cycle ends. The following questions must be answered at this point:

• Is the user satisfied?
• Are the actual resources expenditures versus planned expenditures still acceptable?

Disciplines

In RUP we speak about disciplines. There are nine disciplines in which we categorize our tasks in a software development project according to RUP. First, we have the engineering disciplines:

• Business modeling
• Requirements
• Analysis and design
• Implementation
• Test
• Deployment

Then we have three supporting disciplines:

• Configuration and change management
• Project management
• Environment

Let’s spend a few moments going over these in more detail. This is interesting especially when we compare this lineup to Scrum later in this chapter.

Business Modeling Discipline

The Business modeling discipline is the first one out. The aim of this discipline is to establish a better understanding between business engineering and software engineering. A welcome advance compared to the waterfall approach, because we already have seen that bridging the gap between these two is important. If you turn to Chapter 4 you will see our idea of how this could be done. Business modeling explains how to describe a vision of the organization in which the system will be deployed. It also tells us how to use this vision as a basis when outlining the process as well as when selecting roles and responsibilities.

Requirements Discipline

This discipline is responsible for gathering the requirements and uses these to describe what the system is supposed to do, so that developers and stakeholders can agree on what to build.

Analysis and Design Discipline

This discipline takes the requirements and transforms them into a design of the system. The aim is to have a design that can be changed easily when functional requirements change, which of course they will during the project. The design model is an abstraction of what the source code will look like. We can call it a blueprint if we like, which shows us components, classes, subsystems, and so on. Having well-defined interfaces between components is another important task for this discipline. We also develop descriptions of how objects of the design collaborate to perform the use cases.

Implementation Discipline

The Implementation discipline takes the blueprint and converts it to executable code. We also find testing of developed components as units here. Components developed by individual teams are integrated into an executable system by this discipline. Focus is very much on component-based development, which is a way of developing that encourages reuse of existing components. To be honest, this is a good idea in theory, but in real life I have seen very few good examples of component reuse. Most times reuse is a developer using previously built snippets of code instead of components. It is a good effort to propagate components reuse, but in reality it seems like it is not working.

Test Discipline

There are several purposes with the Test discipline:

• Verifying interaction between objects.
• Verifying all components.
• Making sure all defects are fixed, retested, and closed.
• Verifying that all requirements have been implemented (correctly).
• Identifying defects.
• Making sure defects are addressed.
• Making sure defects are addressed before deployment.

RUP states that testing should be an integrated part of the whole development project, and we cannot agree more. The purpose is to find defects as early as possible, when they can be fixed using minimal effort. There are four quality dimensions by which tests are carried out:

• Reliability
• Functionality
• Application performance
• System performance

Deployment Discipline

The activity in the Deployment discipline needs to be planned for early in the project. Deployment is mostly centered at the end of the Construction phase and the Transition phase, but to successfully deploy an application or system we need to plan for this event earlier on. This discipline focuses on delivering successful product releases. It also focuses on delivering the software to the end users. Included in this work are the tasks of packaging, distributing, and installing the software. Furthermore, the people in this discipline provide help to users, so that deployment runs smoothly.

Configuration and Change Management Discipline

RUP distinguishes three areas within the Configuration and change management discipline. There is configuration management, which is the structuring of products. We also need a way to keep control of our releases and the artifacts belonging to them, and these are tasks belonging to this area. The second area is change request management, where we keep track of all change proposals we receive for the different versions of the software. The third and last area is status and measurement management. When a change request is created it goes through different states in its workflow. It transforms from new, to logged, to approved, to assigned, to completed. This area describes how we can get reports on status of the software and its change requests and different releases. These reports are important for the project team as well as for stakeholders, so that they have a good understanding of the current project status.

Project Management Discipline

As we have seen in the preceding sections we have two dimensions of a project in RUP. We have the four phases and the iterations within them. The Project management discipline focuses on planning the phases, which is done in the phase plan. Planning how many iterations (in the Iteration plan) might be needed and also how to handle risks through the project. This discipline also monitors the progress of the project. There are some things RUP does not include in the project management discipline however. It does not cover managing people, which is usually a project management responsibility. Budget management and contract management also are not included.

Environment Discipline

The Environment discipline is the final discipline in RUP. Contrary to what one might think, we do not include the software environment here. Instead we find the environment for the project—that is, the processes and tools we should use when running the project, what work products we should deliver (more about these in a moment), and so on.

Work Products, Roles, and Tasks

All through the project we have various deliverables that should be produced. RUP called them artifacts originally and that is the term that has stuck in most people’s minds. However, after IBM took over RUP responsibilities, the term work products was coined and we will use this new term from now on.

A work product could be an architecture model, the risk list, the iteration plan, and so on. It is a representation of the results of a task. We find all documents and models we produce in the project under the term.

A task is a unit of work, which provides a meaningful result. A task is assigned to a role. A role in its turn defines a set of related skills, competences and responsibilities.

For example, the work product iteration plan is the result of a task, produce iteration plan, which is performed by the role project manager. Another example could be the task defining the architecture, which produces the result, or work product, architecture model. This is performed by the role called architect.

There are several benefits of using RUP. Many projects have completed successfully using an adaptation of this framework. We would say that project quality is significantly improved by using RUP. The problem however is that RUP has grown to be an almost impenetrable framework. There is too much to consider and choose from which makes it very hard to adapt correctly. One of our colleagues said that he did not like a model that needed adaptation to that extent. And we can understand that. Also the process is too strict and not truly iterative compared to Scrum or any other truly agile methodology. Even Ivar Jacobson, one of RUPs founders, seems to have realized that the framework has grown too immense. He has during the recent years improved on RUP and created a new more agile framework.

Empirical Process Control

So what is this model or framework all about? First, let’s define two ways to solve different problems. We touched on the problems with projects in Chapter 1. When we have a problem that is similar time after time (like road construction, for example or implementing a standard system) we pretty much know what we have to expect of the various tasks at hand. We can then easily use a process, like the Waterfall model, perhaps, that produces acceptable quality output over and over again.¹⁰ This approach is called a defined process control.

When it comes to a more complex problem however, like building a software system, we earlier saw that the traditional models do not work. We then must use something called empirical process control according to Schwaber.¹¹ Empirical process control has three legs to stand on:

• Transparency
• Inspection
• Adaptation

“Transparency means that the aspects of the process that affect the outcome must be visible to those controlling the process.”¹² This means that to be able to approve the outcome we must agree on what the criteria for the outcome are. Two persons cannot say they are “done” with a task unless they both agree what the criteria for “done” are.

The next leg is inspection. The process must be inspected as frequently as necessary to find unacceptable variances in the process. Because all inspections might lead to a need for making changes to the process itself, we also need to revise the inspections to fit the new process. To accomplish this, we need a skilled inspector that knows what he or she is inspecting.

The last leg is adaptation. We saw that an inspection might lead to a change of the process. This is one example of an adaptation. Another can be that we need to adapt the material being processed as a result of an inspection. All adaptations must be made as quickly as possible to minimize deviation further on.

Schwaber ruses the example of code review when he discusses empirical process control. “The code is reviewed against coding standards and industry best practices. Everyone involved in the review fully and mutually understands these standards and best practices. The code review occurs whenever someone feels that a section of code is complete. The most experienced developers review the code, and their comments and suggestions lead to the developer adjusting his or her code.”¹³ Simple isn’t it? I could not have said it better myself.

Complexity in Projects

What makes a software development process so complex anyway? We discussed it a little previously but let us dive a little bit deeper into it here. In theory it might seem pretty straightforward to build software systems. We write code that logically instructs the CPU to control the computer. I mean how hard can it be? Alas, it is not that simple I’m afraid. The persons writing the code are complex machines in themselves. They have different backgrounds, IQs, EQs, views, attitudes, and so on. Their personal life also adds to their complexity.

The requirements might also be complex and they also have a tendency to change over time. According to Schwaber a large percentage of the requirements gathered at the beginning of a software project change during the project. And 60 percent of the features we build are rarely or never used in the end. Many times in my projects several persons are responsible for the requirements at the customer. Quite often they have diverging agendas as to why and what to build. Often the stakeholders have a hard time expressing what they really want. It is when they see a first prototype of the system that they fully start to see the possibilities with the software, and only then can they begin to understand what they want.

Rarely is it the case that just one computer is involved in a system either. Interaction between several machines is mostly the case. We might have a web farm for our GUI, a cluster for our application tier, a backend SQL Server, some external web services and often a legacy system, all needing to integrate to solve the needs of the new system.

When complex things interact—as people, requirements, and technology do in a software project—the level of complexity increases greatly. So it is safe to say that we don’t have any simple software problems anymore. They are all complex. Schwaber realizes this as well, and in Figure 3-9 we can see his complexity assessment graph.

The projects in the anarchy area are chaotic and unworkable. To get them to reach their finish lines we probably need to resolve serious issues before even starting the project.

What Scrum tries to do is address this inherent complexity by implementing inspection, adaptation, and visibility as we previously saw in empirical process control . Scrum does so by having simple practices and rules, as you will now see.

What Scrum Is

Scrum is a powerful, iterative, and incremental process. Many get fooled by its perceived simplicity, but keep in mind that Scrum can handle CMMI at level 5, which not many other processes can do. Figure 3-10 shows the skeleton of the Scrum model to which we attach the rules and practices. Each iteration consists of several daily inspections.

During these inspections, team members evaluate each other’s work and the activities performed since the last inspection. If necessary adjustments (adaptation) are found they are implemented as quickly as possible. The iterations also conclude with inspections when more adaptations can be made. This cycle repeats until it is no longer funded.

All the requirements that are known at the beginning of the project are gathered in the product backlog, which is one of the artifacts of Scrum. We will soon come back to this. The project team reviews the backlog and selects which requirements should be included in the first iteration, or sprint as it is called in Scrum. These selected requirements are added to the sprint backlog where they are broken down into more detailed items. The team then makes their best effort to turn the sprint backlog into a shippable increment of the final product. The team is self-managing, which means they collectively decide who does what and what is the best way to solve the problems.

The increment is presented to the stakeholder(s) at the end of the sprint so they can inspect it and make any adaptations necessary to the project. The sprint is most often 30 days, although often I have seen sprints somewhere between 2-4 weeks in many projects. It depends a bit on the sprint backlog items. When I took my Scrum master certification, Ken Schwaber related that he once had had to have a one-week sprint in a project. The reason for this was that the team malfunctioned and this way he could more easily catch the reason for this and adjust the process so that the project ran more smoothly.

The stakeholder’s adaptations and feedback are put into the product backlog and prioritized again. Then the team starts the process all over again and selects the backlog items they think they can finish during the next sprint. These are put into the sprint backlog for the next sprint and broken down to more manageable items. And so it continues until the stakeholder thinks they have received the business value they wanted and funding stops.

If we look at the three legs of empirical process control again we can see that Scrum covers them nicely. Transparency is implemented by letting the team and stakeholders agree on what is the expected outcome of the project and of each iteration. Inspection occurs daily and also at the end of each sprint. Adaptations are the direct result of these inspections and a necessary thing in Scrum.

The Roles in Scrum

There are different roles in Scrum just as there are in all previously mentioned models. But the difference is that Scrum has fewer roles, which are not defined in the same strict way as in the others. Scrum has the three roles:

• The product owner
• The team
• The Scrum master

The Product Owner

Let’s start with the product owner. He or she is responsible to those funding the project to deliver a product or a system that gives the best Return On Investment (ROI) they could get from the project. The product owner must acquire the initial funding of the project and also make sure it is funded through its lifespan. The product owner represents everyone with a stake in the project and its result. At the beginning of a project the product owner gathers the initial requirements and puts these into the project backlog. It is the product owner who ultimately decides which requirements have the highest priority based on ROI or business value (for example) and decides into which sprint they should go. During the project the product owner inspects the project and prioritizes the product backlog and sprint backlogs so that the stakeholders’ needs are met.

The Team

The team is responsible for the development. There are no specific roles in the team. Because the team is cross-functional and self-organizing it is their responsibility to make sure they have the competencies and staff they need for solving the problems. So it is not the scrum master who decides who does what and when, as a project manager would do in a traditional approach. These are some of the reasons behind this thought, as taught by Ken Schwaber in his scrum master course:

• People are most productive when they manage themselves;
• People take their commitment more seriously than other people’s commitment for them (like when a project manager commits that the person should accomplish something);
• People have many creative moments during down time;
• People always do the best they can; and,
• Under pressure to work harder, developers automatically and increasingly reduce quality.

The team should be seven persons plus or minus two for optimal result. A logical team consists of one programmer, one tester, a half time analyst/designer, and a half time technical writer. The optimal physical team has 2.5 logical teams. The team decides which items in the backlog they can manage for each sprint based on the prioritized backlog. This whole thinking is a giant leap from traditional project management and takes some getting used to. Some persons do not accept this and find it impossible to work this way.

The Scrum Master

The scrum master is responsible for the Scrum process and has to make sure everybody in the team, the product owner, or anyone else involved in the project knows and understands the process. The scrum master makes sure that everyone follows the rules and practices of the Scrum process. So the scrum master does not manage the team, the team is as we saw self-managing.

If a conflict occurs in the team, the scrum master should be the “oil” that helps the team work out their problems smoothly. It is also the scrum master’s responsibility to protect the team from the outside world so that they can work in peace and quiet during the sprint, focused on delivering business value. The following lists the scrum master responsibilities again according to Ken Schwaber’s course material:

• Removing the barriers between development and the customer so the customer directly drives development;
• Teaching the customer how to maximize ROI and meet their objectives through Scrum;
• Improving the lives of the development team by facilitating creativity and empowerment;
• Improving the productivity of the development team in any way possible; and,
• Improving the engineering practices and tools so each increment of functionality is potentially shippable.

The Scrum Process

Now that we know the basics of Scrum it is time to take a look at what happens during a Scrum project.

The product owner, after arranging initial funding for the project, puts together the product backlog by gathering functional as well as non-functional requirements. The focus is to turn the product backlog into functionality and it is prioritized so that the requirements giving the greatest business value or having the highest risk come first. Remember that this approach is a value-up paradigm where we set business value first.

Then the product backlog is divided into suggested releases (if necessary), which should be possible to implement immediately. This means that when a release is finished we should be able to put it into production at once so that we can start getting the business value as quickly as possible. We do not have to wait until the whole project is done until we can start getting return on our stakeholder’s investments.

Because the Scrum process is an adaptive process, this is just the starting point. The product backlog and the priorities change during the project as business requirements change and also depending on how well the team succeeds in producing functionality. The constant inspections also affect the process.

When a sprint is starting, it initiates with a sprint planning meeting. At this meeting the product owner and the team decides, based on the product owner’s prioritization, what will be done during this sprint. The items selected from the product backlog are put into the sprint backlog.

The sprint planning meeting is time-boxed and cannot last more than eight hours. The reason for this strict time-box is that the team wants to avoid too much paperwork about what should be done.

The meeting has two parts. The first four hours are spent with the team and the product owner, where the latter presents the highest priority product backlog issues and the team questions him/her about them so that they know what the requirements mean. The next four hours are used by the team so that they can plan the sprint and break down the selected product backlog items into the sprint backlog.

When the project is rolling, each day starts with a 15-minute daily scrum or stand up meeting (see Figure 3-11). This is the 24-hour inspection. During this meeting each team member answers three questions:

• What have you done since the last daily scrum meeting?
• What do you plan to do on this project until the next daily scrum (your commitments)?
• What impediments are in the way of you meeting your commitments toward this sprint and this project?

The reason for this meeting is to catch problems and hence be able to make timely adjustments to the process. It is the scrum master’s responsibility to help the team members get rid of any impediments they may have.

When a sprint comes to an end a sprint review is held. This meeting is also time-boxed, but at four hours instead of eight. The product owner and the stakeholders can get a chance to see what the team has produced during the sprint and reflect on this. But it is important to remember that this meeting is not a demonstration, it is a collaborative meeting between the persons involved.

Now there is only one meeting left; the sprint retrospective. The sprint retrospective takes place between the sprint review and the next sprint planning meeting. It is time-boxed at three hours. The scrum master encourages the team to adjust the development process, still within the Scrum process and practices framework boundaries, so that the process can be more effective for the next sprint.

What happens if we have a larger project than one with only a team of approximately seven persons? Can Scrum scale to handle this? According to Mike Cohn in an article on the Scrum alliance web site,¹⁴ we can use a process called scrum of scrums:

By using this technique we can scale Scrum infinitely, at least in theory.

That’s basically it. Scrum is a lean process and appeals a great deal to us. One of the authors (Rossberg) has had the privilege to do his scrum master certification during a course held by Ken Schwaber and this was a very uplifting experience. Unfortunately, some customers or stakeholders can find Scrum a bit vague, so they won’t try it. They think they have more control the way they used to run projects and are perhaps a bit afraid to embrace this modern way of doing projects. This still has not changed over the years, although more and more people we meet have seen what scrum and agile can do to help them run better projects.

Some even think that documentation and planning are not necessary in Scrum. Developers like this idea because they don’t want to write documents, while stakeholders tremble at the thought. But nothing could be further from the truth. Scrum does not say we do not document or plan. The contrary is true. Planning, for instance, is done every day, during the daily scrum (see Figure 3-12). Documents should also be written, but we scale away documents that are not necessary—documents that are produced only for the sake of documentation and almost never are read after they are produced. We document what is needed for the system and the project. We document our code; we document traceability, and so on.

We have found that some companies think they are using Scrum just because they develop iteratively. In many cases they have changed the Scrum process so that it will not help them solve their development problems, problems that are clearly visible in a true Scrum project. Instead they have used Scrum as make-up covering the bad spots and when the project still fails, they argue that Scrum doesn’t work either, we still do not deliver value, or still have over runs, and so on. So when implementing Scrum follow the process and framework, adjust the organization to Scrum not the Scrum process to the organization.

How about architecture, the IT architect shouts. Don’t we need to set the architecture before we start coding? Well, no! Scrum uses the concept of emerging architecture, which means we do a base architecture during the first sprint. This is evolved and built upon as the sprints continue. So the architecture is emerging as we go along, giving us agility in this concept as well.

When a customer is hesitating, we must use our most persuasive skills to get them to at least do a pilot on a project and see how it goes. It is our firm belief that most will be pleasantly surprised afterward. When they see that they get a product better meeting their requirements and giving better ROI in a shorter time than traditional projects, they usually melt.