What to Measure

You can measure many aspects of your software products, projects, and processes. The trick is to select a small and balanced set of metrics that will help your organization track progress toward its goals. Goal-question-metric (GQM) is an effective technique for selecting appropriate metrics to meet your needs (Daskalantonakis 1992; Basili and Rombach 1988). With GQM, you begin by selecting a few project or organizational goals. State the goals to be as quantitative and measurable as you can. They might include targets such as the following:

  • Reduce maintenance costs by 50 percent within one year.

  • Improve schedule estimation accuracy to within 10 percent of actuals.

  • Reduce system testing time by three weeks on the next project.

  • Reduce the average time to close a defect by 40 percent within three months.

For each goal, think of questions you would have to answer to judge whether your team is reaching that goal. If your goal was "reduce maintenance costs by 50 percent within one year," these might be some appropriate questions:

  • How much do we spend on maintenance each month?

  • What fraction of our maintenance costs do we spend on each application we support?

  • How much do we currently spend on adaptive maintenance (adapting to a changed environment), perfective maintenance (adding enhancements), and corrective maintenance (fixing defects)?

Finally, identify metrics that will provide answers to each question. Some of these will be simple items you can count directly, such as the total budget spent on maintenance. These are called base measures. Other metrics are derived measures, which are calculated from one or more base measures, typically as simple sums or ratios. To answer the final question in the previous list, you must know the hours spent on each of the three maintenance activity types and the total maintenance cost over a period of time.

Note that I expressed several goals in terms of a percentage change from the current level. The first step of a metrics program is to establish a current baseline, so you can track progress against it and toward your goals. I prefer to establish relative improvement goals ("reduce maintenance by 50 percent") rather than absolute goals ("reduce maintenance to 20 percent of total effort"). You can probably reduce maintenance to 20 percent of total organizational effort within a year if you are currently at 30 percent, but not if you spend 80 percent of your effort on maintenance today.

Your balanced metrics set should eventually include items relating to the six basic dimensions of software measurement: size, time, effort, cost, quality, and status. Table 12-1 suggests some metrics that individual developers, project teams, and development organizations should consider collecting. Some of these metrics relate to products, others to projects, and the remainder to processes. You should track most of these metrics over time. For example, your routine project tracking activities should monitor the percentage of requirements implemented and tested, the number of open and closed defects, and so on. You can’t start with all of these at once, but I recommend including at least the following measurements early in your metrics program:

  1. Product size: Count lines of code, function points, object classes, requirements, user stories, or GUI elements. Adjust raw counts for complexity.

  2. Estimated and actual duration: Count in units of calendar time.

    Table 12-1. Appropriate Metrics for Software Developers, Teams, and Organizations

    Level

    Appropriate Metrics

    Individual Developer

    1. Work effort distribution

    2. Estimated and actual task duration and effort

    3. Code covered by unit testing

    4. Number and type of defects found by unit testing

    5. Number and type of defects found by peer reviews

    Project Team

    1. Product size

    2. Estimated and actual duration between major milestones

    3. Estimated and actual staffing levels

    4. Number of tasks planned and completed

    5. Work effort distribution

    6. Requirements status

    7. Requirements volatility

    8. Number of defects found by integration and system testing

    9. Number of defects found by peer reviews

    10. Defect status distribution

    11. Percentage of tests passed

    Development Organization

    1. Overall project cycle time

    2. Planned and actual schedule performance

    3. Planned and actual budget performance

    4. Schedule and effort estimating accuracy

    5. Released defect levels

    6. Reuse effectiveness

  3. Estimated and actual effort: Count in units of labor hours.

  4. Work effort distribution: Record the time spent in various development and maintenance activities (Wiegers 1996).

  5. Defects: Track the number, type, severity, and status of defects found by testing, peer reviews, and customers.

  6. Requirements status: Track the number of requirements that are proposed, approved, implemented, verified, deleted, and rejected.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.17.157.6