Chapter 11. Managing Agile Projects

Many software projects are moving to Agile methods. This chapter is not about teaching Agile development; you can read about Agile in many articles and books. Instead, it is about realizing Agile with Rational Team Concert (RTC) and ClearQuest.

While Rational Team Concert was developed with Agile methods in mind, Rational ClearQuest is an older product and was developed with more traditional methods in mind. Luckily ClearQuest is highly customizable, so we can develop schemas with a process that meets the modern development environment.

In this chapter we shall explain in brief the Agile method concept, just to set the right context for the rest of the chapter. The Scrum process is becoming more popular with all types of software development projects. We will use RTC’s process enactment of Scrum to illustrate some of the methods within this Agile process. We shall later explain how to build a schema with ClearQuest that will help Agile teams and stakeholders manage their projects smarter to help produce better products.

ClearQuest users using Application Lifecycle Management (ALM) will learn how to configure a project to meet the needs of Agile teams.

11.1. Defining Agile Development

There is no single definition of what Agile development is. There are some principles that many agree upon, and teams can adapt them as it suits their organization. Scott Ambler (see “Resources” at the end of this chapter) defines Agile software development as follows:

• Agile is an iterative and incremental (evolutionary) approach to software development

• which is performed in a highly collaborative manner

• by self-organizing teams

• with “just enough” ceremony

• that produces high quality software

• in a cost effective and timely manner

• which meets the changing needs of its stakeholders.

We shall later see how both Rational Team Concert and a ClearQuest schema we propose respond to this definition.

The Agile system development lifecycle is described in Figure 11.1.

11.2. Agile and Scrum in a Nutshell

Scrum is a framework for managing Agile software development projects. A Scrum project starts with a definition of the items that the system should include and address, including functionality, features, and technology. These items are often called requests. The list of requests is called the product backlog (or project backlog). The requests can be of various types: textual requirements, use cases, test cases, stories, defects, enhancement requests, and so on. The requests are likely captured in an external tool such as Rational Requirements Composer (RRC) and are referenced or linked from the change management tool.

Requests can be submitted by any team member, affected users, or stakeholders. The request content is elicited from various sources, but the requests are prioritized by the product owner only and not by the development team. The product backlog is dynamic; it changes and evolves as the project advances.

During the first phase of the project, often called warm-up, the requirements are analyzed by SMEs to determined feasibility, cost, effort, and risk. This phase is done with the close participation of the stakeholders. The analysis gives the stakeholders a better basis for setting their priorities.

Agile projects are divided into iterations; the iteration has a fixed duration of a few weeks, usually two to four. In Scrum the iteration is called a sprint.

The Scrum team’s responsibility is to take the requirements from the backlog and develop a usable product or component that has real value to the project, within the sprint period. The team is cross-functional and performs all activities related to the delivery. The team as a whole is responsible for delivering the requirements.

The sprint starts with a team planning meeting. Each team takes one or more top-priority requests from the product backlog, as many as they think they can develop and deliver during the sprint. A sprint must finish with delivery of a new, executable product; thus each sprint consists of all lifecycle disciplines: design, development, testing, and so forth.

The team creates a list of activities (or tasks) from the requirement(s) they have selected. This is called the sprint backlog. The activities represent the way the team decides to implement the requirements. Each activity is assigned to a team member (or sometimes to team pairs).

Each Scrum team is autonomous; the team decides how to develop work products from a requirement. Teams are organized in a way that they could be autonomous, thus having expertise in various domains. The team decides which of the Agile development methods to use—XP, pair programming, test-driven development, or another method. They decide how much modeling to do. However, the team must adhere to regulations, standards, and rules that the organization has set.

Figure 11.1. The Agile System Development Lifecycle
(copyright Scott W. Ambler)

An important role within the Scrum process is the Scrum Master. This role facilitates and guides the teams in adopting the agreed-upon practices and removes impediments. The Scrum Master is not part of the team and does not give orders to the team.

Every day the team gathers for a short (15-minute) stand-up meeting called a Daily Scrum Meeting. The purpose of the meeting is to share status and identify potential issues. During the Daily Scrum Meeting the team progress is reviewed and impediments are identified for removal by management. The Scrum Master facilitates the daily meetings, tracks progress, and works to resolve any inhibitors raised by team members.

Each Scrum member briefly reports on three items:

• What was accomplished since the previous meeting?

• What is planned to be accomplished before the next meeting?

• What prevents the member from accomplishing the activities?

The report should focus on information that helps other team members gain knowledge, learn a lesson, or contribute from their experience. Important practices in the meeting are honesty and transparency.

At the end of the sprint, the team is gathered for a Sprint Review Meeting with the stakeholders. The team reviews the work that was completed and that was not completed. The products developed are demonstrated to the stakeholders to examine their value. They will make a decision about whether to make use of the products and obtain funding for the next iteration. Additional elements of the meeting are to learn from the experience in order to make an improvement for the next iteration. Some teams conduct a different meeting for that purpose; members give their opinions on what went well and what needs improvement. This meeting is called a Sprint Retrospective.

The Sprint Review usually results in some adaptation of the product backlog. Enhancement requests are added, defects are submitted, and maybe new features are introduced. The remaining items in the sprint backlog are moved to the product backlog. The team can add an activity, for example, to learn and experiment with a new technology, or to perform more performance tests. Stakeholders and product owners may decide to change the priorities of some backlog requirements.

Now a new sprint starts again with teams selecting top-priority requests for development. The sprint cycles continue until the stakeholders think they have enough value and quality to release a product. This stage is called the Release Iteration or the End Game. During this iteration final system testing and acceptance testing are performed. Stakeholders may request some defect fixing. The team finalizes system and user documentation. Users and administrators are trained, and the system is deployed to production.

The Scrum project process is often described using the schema illustrated in Figure 11.2.

Figure 11.2. The Scrum Project Lifecycle
(copyright Scott W. Ambler)

11.3. Realization with Rational Team Concert

Out of the box, Rational Team Concert has a process template that enacts the Scrum process. This chapter will discuss what Scrum is in terms of how Rational Team Concert realizes it. By using a tool’s out-of-the-box process, your organization can be more Agile, as you will spend less time designing, creating, and testing a custom solution. Within Rational Team Concert, the Scrum project starts with a definition of all the items that the system should include and address, including functionality, features, and technology. These items are called stories. Rational Team Concert has built in an Agile planning feature through its Web interface. The lists of stories are presented in a product backlog plan. This plan interface is the main focus during the team planning meeting to decide what to work on in the first sprint. The product backlog list is shown in Figure 11.3.

Figure 11.3. The product backlog of Release 2.0

The user stories can be submitted by any team member, affected users, or stakeholders. They are submitted against the backlog category and will show up in the product’s backlog plan and are not assigned to any individual yet. The story content is elicited from various sources. A work item of type Story is shown in Figure 11.4.

The product backlog is dynamic; it changes and evolves as the project advances. Certain items may become more important than others during a sprint. Sometimes this process of prioritizing is called ranking. With Rational Team Concert you can easily drag stories around to rank them relative to their position in the list. You also get a visual display of unranked items. All stories should be ranked to make sure they are not missed. A product backlog ranked list is shown in Figure 11.5.

Figure 11.4. The Story work item

Figure 11.5. Ranking user stories

Within Rational Team Concert you have the ability to attach complexity of effort to a story. The attribute of the Story work item used to convey complexity is Story Points. The larger the Story Points, the more difficult it will be to implement the Story.

In Rational Team Concert you can set up multiple teams working on multiple releases; each release plan has its own current sprint (see Figure 11.6).

Figure 11.6. Sprints of different releases

Figure 11.7 shows the backlog and the sprints (iterations). The backlog contains a list of work items (in this figure of type Story) that should be assigned to teams in a specific sprint.

Figure 11.7. Release 2.0 backlog showing iterations

In Rational Team Concert you facilitate sprint planning meetings by simply dragging and dropping a story from the product backlog to your sprint; this is shown in Figure 11.8.

Figure 11.8. Assigning stories from the product backlog to a sprint

Rational Team Concert can easily show a plan of any iteration. Once a team creates its Sprint backlog, it can be easily communicated and worked further through the Web planning interface of Rational Team Concert. Figure 11.9 shows sprint backlog stories ordered by priority.

Through the use of Rational Team Concert’s Agile planning through the Web interface, you can easily create child tasks from a story. You can drag and drop tasks to make them children of stories, and promote and demote them as needed; this is demonstrated in Figure 11.10.

Figure 11.9. The sprint backlog

Figure 11.10. Working with the sprint backlog

In order to make sure work gets done, you want to assign tasks to individuals. Rational Team Concert supports Agile assignment of work through drag and drop, making it easier to use the tool to run a sprint planning meeting. As you are assigning work to people, you can also easily enter the amount of task time remaining. This is used to calculate the burndown metric. This work breakdown view of the backlog and the progress bar is shown in Figure 11.11.

Within Rational Team Concert, the Scrum Master role and the other Scrum roles are part of the out-of-the-box process supporting Scrum as shown in Figure 11.12. The administrator can define new roles as required.

Figure 11.11. Assigning tasks to individuals

Figure 11.12. Process roles in Scrum

In Rational Team Concert’s Scrum process, impediments are implemented as work items and follow a workflow of open → resolved, so they can be submitted, assigned, and managed just like any other work item. Rational Team Concert also provides another useful view for Agile planning through the Web interface called the Developer Taskboard. This view is perfect for the Daily Scrum Meeting as you can easily see what team members are working on, what they have completed, and other data. The Developer Taskboard view is shown in Figure 11.13.

Figure 11.13. Using the Developer Taskboard in the Daily Scrum Meeting

Another Agile communication vehicle within Rational Team Concert is the dashboard. This is a way that stakeholders and executives can keep informed on the progress of any project. Much of the information contained within Rational Team Concert can be presented easily through feeds to a dashboard. Any Rational Team Concert user (assuming the user has authorization) can set up a dashboard to collect and present items of interest. A sample Rational Team Concert dashboard is shown in Figure 11.14.

Figure 11.14. Project dashboard showing real-time status of events of interest

One of the types of work items that Rational Team Concert supplies out of the box in the Scrum process is a retrospective. You use this work item to log and track the issues that you discuss during the Sprint Retrospective.

11.4. Realization with ClearQuest

In section 11.2, “Agile and Scrum in a Nutshell,” we described the Agile process and the Scrum in particular. Unlike Rational Team Concert, which has a built-in process template to support Scrum, ClearQuest does not have an Agile built-in schema. In this section we shall explain how to build a schema to support Agile projects. Reading the process description in section 11.2, we can identify data objects and workflow scenarios. Now let us take out the data objects from the description. These are

• Request

• Product backlog of requests

• Sprint

• Activity

• Sprint backlog of activities

• Team

• Iteration

Let’s discuss each of these data objects a bit more.

• A request (or CR or any other name that may fit your environment) is realized by a state-based record type. We shall describe the fields of this record type in the next section, but one important field should be the RequestType. Types could be Defect, Enhancement, Feature, Story, Test Case, and so on. Another important field is Priority; the team will select Requests to implement in the iteration based mainly on priority.

• The product backlog of requests and sprint backlog of activities are lists of work items. We do not need to create a special object for these; the reason is that the logs will be realized with queries whose result set is the backlog. The project backlog is a query on the Request record type that filters all of the analyzed requests (ready to be selected for the sprint), sorted by priority. The sprint backlog is a query on the Activity record type that filters all of the opened activities (not completed) that are referenced from a specific sprint, sorted by priority.

• A sprint is realized by a state-based record type. This record type will have fields of type Reference_List to the Requests and the Activity record types, and fields of type Reference to the Team and Project record types.

• An activity (or task or work item or any other name that may fit your environment) is realized by a state-based record type. As stakeholder requests are usually high-level and nontechnical, the team will break down requests into activities. Each activity will be assigned to a team member.

• The team is realized by a stateless record type. The Team record type contains a list of team members, specific roles in the team such as Team Leader, and users having a role.

• Iteration is realized by a stateless record type. The Iteration record type contains the iteration name, the start date, and the end date.

The record types and their relationships are described in Figure 11.15.

We have realized some of the data objects with state-based record types and some with stateless record types, and we have realized the product backlog and the sprint backlog with queries.

Figure 11.15. Entities relation diagram for the Scrum schema

11.4.1. Required Data

In this section we describe the fields for each record type that are essential for the solution. It is assumed that each implementation will include additional fields based on products developed, organizational culture, regulations, and other considerations.

Table 11.1 describes the suggested fields for the Request record type.

Table 11.1. Request Suggested Fields

Figure 11.16 is a screen shot of the Request record, with several of the fields listed in Table 11.1.

Figure 11.16. Request record: main tab

Table 11.2 describes the suggested fields of the Sprint record type.

Table 11.2. Sprint Suggested Fields

Figure 11.17 displays the Sprint record main tab; it shows the sprint details, the responsible team, iteration name, end date, and other information. You can see the list of requests that will be realized by the team in this sprint, in this case one new feature to develop and one defect to fix.

Figure 11.17. Sprint record: main tab

Table 11.3 describes the suggested fields of the Activity record type.

Table 11.3. Activity Suggested Fields

Figure 11.18 is a screen shot of the Activity record main tab, with several of the fields listed in Table 11.3.

Figure 11.18. Activity record: main tab

Figure 11.19 displays the Activities tab in a Sprint record. The team has created five activities of different types, to realize the two requests that are shown in the Requests field.

Figure 11.19. Activities and Requests related to a sprint

We have seen in this section the record types and the fields that construct the Scrum schema.

11.4.2. Understanding the Workflows of Each Record Type

After creating the three state-based records types and the fields in each one, we need to define the state machine for each record type. In the next section we describe the workflow for each record type.

11.4.2.1. Request

The workflow for the Request record type depends a lot on the organization, the stakeholders, and the regulations enforced. We propose the flow shown in Figure 11.20.

Figure 11.20. The Request record type workflow

The Request is submitted to the New state and analyzed to identify feasibility, effort, priority, possible impacts, risks, and so on. After it is analyzed and priority is given by the stakeholders, it can be picked by a team to be developed in a sprint. At the end of the sprint during the Retrospective (review meeting) the deliverables are evaluated. If the deliverables are found to meet the stakeholder request and have the desired quality, the Request can be moved to the Delivered state.

In some cases a Request can be moved from the New state directly to the InSprint state, for example, in the case of a defect or a request submitted by the team. In either case the Request must get a priority value by the stakeholder.

11.4.2.2. Activity

The workflow for the Activity record type is similar to the CMBaseActivity record type of the UCM package. We suggest that this record type be integrated with your version control system. If you are using ClearCase, add the UCM package to the schema and enable the Activity record type to the UCM package. The Request and the Sprint record types should not be UCM-enabled because artifact changes are controlled with the Activity record.

The state machine includes four consecutive states: Waiting, Ready, Active, and Complete (see Figure 11.21).

Figure 11.21. The Activity record type workflow

11.4.2.3. Sprint

The Sprint record type is used for project management as explained in the previous section. During the sprint planning meeting (or before) the record is created and its state is Submitted. When the sprint starts (the sprint iteration start date is reached), the team performs the action StartSprint which moves the record to the state InSprint. When the sprint ends (the sprint iteration end date is reached) and during the sprint Retrospective meeting the team performs the action Close, which moves the record to the state Closed. So the Sprint record type has three consecutive states: Submitted, InSprint, and Closed (see Figure 11.22).

Figure 11.22. The Sprint record type workflow

In this section we have described the suggested workflow for each of the record types that construct the Scrum schema.

11.4.3. Understanding Metrics in Agile Development

We discussed metrics in detail in Chapter 9, “Metrics and Governance”; we include here just a few words on metrics in Agile projects. Metrics measure data of direct business value to the organization. In the repetitive cycles of Agile projects, improvement can bring value, and to improve we need to measure. Working with ClearQuest, we can measure by means of charts and reports provided by the tool.

Figure 11.23 is a screen shot of the ClearQuest client displaying some typical queries and charts for Scrum projects in the workspace. The executed chart shows Planned Effort, and it displays the estimated effort of requests in each iteration.

Figure 11.23. Planned Effort for Iteration chart

The “classic” release burndown charts and velocity charts cannot be created with the ClearQuest chart wizard, but you can create reports and distribution charts that display the closed activities per iteration, or the actual/estimated effort of activities in an iteration, which will provide similar information. Also, using tools such as Rational Insight will allow you to present burndown and velocity charts.

11.5. Agile with the ALM Schema

Companies that have decided to adopt the CQ-ALM (ClearQuest Application Lifecycle Management) schema on an enterprise-wide basis may still need to deal with projects and teams that are using an Agile development method. Those companies may ask if they need an additional schema for these teams. Our answer is that they do not; they can use the ALM schema. We shall explain how they can configure an ALM project in an Agile way. We assume that you are familiar to some degree with the CQ-ALM and the ALM terminology. Section 12.4, “ClearCase, ClearQuest ALM, Build Forge Integrated Solution Architecture,” in Chapter 12, “Sample Applications and Solutions,” includes additional discussion and examples of the CQ-ALM schema.

The ALM schema includes three work management state-based record types: Request, Task, and Activity. The Request record is similar to the one described in the previous sections. A Task record is created when the request is approved for development. Priority behavior is mandatory and you should add a field of type Integer for the estimated effort unless you are using CQ-ALM 1.1 where this field is already provided. Approving the request means that it was analyzed by both the technical and the business teams, and it was prioritized by the product owner or the stakeholders. The product backlog is a list of tasks generated by a query that displays all the opened tasks of a given project sorted by priority.

The ALM schema defines project phases and iterations. Agile projects do not use phases, so the ClearQuest admin can create a single phase record and name it with a dummy name such as Iteration or Sprint. The next step is to create iteration records with the numeric values of the iteration. For example, create iteration records and label them with 1, 2, and so forth. Using numeric values is only a suggestion; you can use any method, such as week in the year. When using the system, you will have Iteration 1, Iteration 2, or Sprint 1, Sprint 2, and so forth.

Figure 11.24 shows an ALMTask record assigned to Sprint 2.

Another differentiator between our Agile schema and the ALM schema is the use of roles. Agile projects usually adopt the whole team practice; the skills of the whole team are what matters and team members’ roles are less relevant. So we suggest creating a Role Label record for each team, using names like Team-A, Team-B, and so forth. In the Members tab add the team members to the Members field and the team leader to the Primary field. If relevant to your project add additional role labels such as TeamLeader or ScrumMaster (see Figure 11.25).

In the Scrum schema described in the previous sections we used a record type called Sprint. Do we need to create such a record type in the ALM schema? The answer is no.

Figure 11.24. ALMTask record assigned to sprint (iteration)

Figure 11.25. ALMRole record assigned to an Agile team

A sprint is a team effort for a given iteration. What we need to do is to link the Request, Task, and Activity to the team and the iteration (see Figure 11.26). Among the many possible solutions we shall mention three:

• Add a field called Sprint to each of the Request, Task, and Activity record types. The team will fill in a value that uniquely identifies the sprint. The field value should be automatically copied when a child record is created. Although this solution is simple, it requires a schema change.

• The second solution uses existing fields. The Task record already has the field Iteration that references the ALMiteration record that has fields such as start_date, end_date, status_label, description, and others. The Activity and the Request record types have references to Task, so for each record we can find the iteration that the record was assigned to. We also need to relate the record to the team working on it; this is done using the ALMRole record as previously explained.

• The third solution is similar to the second one. We previously explained that we gave the phase a dummy value, so instead of using a dummy value we can set the phase name to be the team name. Now we have for the iteration a meaningful value that is the team name and the iteration name. The user will see in the Iteration field Team-A 1, Team-A 2, and so on, which identify the sprint numbers for Team-A.

Figure 11.26. ALMTask record assigned to a team

The team (field Roles) in Figure 11.26 is automatically selected when the task type is selected. This is defined by creating an ALMWorkConfiguration record with the following fields: Project, Record Type, Type Label, Roles.

The three solutions allow you to create queries to see the sprint status, cumulative effort, status of each activity, defects reported against a specific sprint, and other sprint queries, charts, and reports as required.

During the sprint planning meeting the team creates one or more activities for each task. Each activity is assigned to a team member. The number of child activities that will be created per task and the types of those activities can be defined and set by each team in the ALMWork-Configuration record. This is a powerful and useful feature of the ALM schema.

In the Sprint Review Meeting, if the team found the tested deliverables to have the required quality, the team can Complete (an action) the Activated (a state) Tasks. Now the stakeholder can Accept (an action) the related Requests to release the deliverables and to Close the sprint (see Figure 11.27).

Figure 11.27. ALMRequest solution is accepted by the requestor

The ALMRequest record is accepted by the requestor (stakeholder) after the ALMTask is completed with resolution code Completed.

11.6. Resources

11.6.1. Materials by Scott Ambler

Ambler, Scott, “Agile Modeling—Effective Practices for Modeling and Documentation,” www.agilemodeling.com, 2007 (accessed February 23, 2011).

Ambler, Scott, “Scott Ambler’s Articles and Other Writings,” www.ambysoft.com/onlineWritings.html, 1997–2011 (accessed February 23, 2011).

This page provides links to books and Web-based writings.

Ambler, Scott, “The Agile System Development Life Cycle (SDLC),” www.ambysoft.com/essays/agileLifecycle.html, 2005–2010 (accessed February 23, 2011).

11.6.2. DeveloperWorks Articles

Pampino, Carolyn, and Robert Pierce, “Application Lifecycle Management with Rational ClearQuest 7.1.0.0,” IBM developerWorks, www.ibm.com/developerworks/rational/library/edge/08/mar08/pampino-pierce/, 2008 (accessed February 23, 2011).

Ellingsworth, Millard, and Thomas Starz, “Scrum Project Management with IBM Rational Team Concert Version 2,” IBM developerWorks, www.ibm.com/developerworks/rational/library/09/scrumprojectmanagementteamconcert-1/index.html, 2009 (accessed February 23, 2011).

Lee, Kevin A., “Agile SCM and the IBM Rational Toolset,” IBM developerWorks, www.ibm.com/developerworks/rational/library/jun06/lee/index.html?S_TACT=105AGX15&S_CMP=EDU, 2006 (accessed February 23, 2011).

11.6.3. Other Information

“Agile Alliance,” www.agilealliance.org/, 2011 (accessed February 23, 2011).

“OpenUP,” http://epf.eclipse.org/wikis/openup, 2011 (accessed February 23, 2011).

“Scrum Alliance,” www.scrumalliance.org/, 2011 (accessed February 23, 2011).

Wells, Don, “Extreme Programming: A Gentle Introduction,” www.extremeprogramming.org, 2009 (accessed February 23, 2011).

Wikipedia, “Lean Software Development,” http://en.wikipedia.org/wiki/Lean_software_development, 2009 (accessed February 23, 2011).

Schwaber, Ken, and Mike Beedle, Agile Software Development with Scrum (Upper Saddle River, NJ: Prentice Hall, 2002).

11.7. Summary

In this chapter we started with a short explanation of Agile principles and continued with a somewhat more detailed description of the Scrum method. We later explained how Scrum is realized in Rational Team Concert using the provided Scrum process template.

We continued with an explanation of how to create a ClearQuest schema to manage Agile projects and specifically Scrum projects. It is important to mention that the proposed solution can be modified and adapted to each company or project. The same principles can be applied to extend existing schemas. Some examples of metrics derived with ClearQuest charts are explained.

The ClearQuest schema described will be available to download from the IBM developerWorks site. The schema serves as a skeleton and does not pretend to be a complete solution. Use it as a basis for your schema; add fields and hooks to create rules and to automate operations.

In the last section we explained how organizations that use the built-in CQ-ALM schema can configure an ALM project to support Agile teams. We proposed several solutions that require only minimal modifications or no modifications to the schema.