76 ◾ Simple Statistical Methods for Software Engineering
ere is a dierence between goal and need.
Goal: the end toward which eort is directed
Need: a condition requiring supply or relief
Goals are complex; they consist of interconnected layers and are influenced by
“personal goals” and “self-efficacy.” Goals have multiple dimensions, are likely to
drive discussions of metric design into inconclusive divergence, and are correspond-
ingly undesirable to rely on for deriving metrics. Needs have a simple structure and
are well defined with greater degree of objectivity in software development projects.
Meaning of Metrics: Interpreting Metric Data
We define a metric by defining the relationship the metric has with raw data. Metric
definition inheres the meaning of that metric. e defining equation is more signifi-
cant than the name we give to a metric. Names could mislead, but definitions do not.
It is good to recall metric definitions, even if obvious, before beginning interpretation.
Having recalled the metric definition, we now look at metric data. It is better to
work with a data table that shows the raw data and the derived metric in separate
columns. Visibility into basic observations helps in getting a detailed understand-
ing of metrics. In one column, we can have the time stamp of data. If data fall into
categories, it is better to include the category name in one column. Occasionally, we
may have to allocate additional columns to accept further categorization schemes.
Next we should construct a box plot of metric data and analyze the statisti-
cal nature of data. e box, whiskers, and outliers seen in the box plot must be
understood and explained. Questions regarding the stability and the reasonable
dispersion of the metric must be addressed. We can support this enquiry with a
descriptive statistics analysis.
Data have intrinsic meaning that can be seen by applying statistical, engi-
neering, and management perspectives.
Box 5.2 olyMPic RunneR
Time in a school’s final sprint competition is measured using an analog stop-
watch. In interschool competitions at the state level, time is measured more
precisely using a digital stopwatch. In Olympic sprints, time is measured
by laser systems controlled by computers to the precision of a millisecond.
As the capability of running improves, the precision of measurement also
is improved. Likewise, when software engineering practices become more
mature, metric capability also is improved. e quality of metric data also is
improved. Metrics and maturity go together.
Deriving Metrics ◾ 77
e engineering and management understanding of the “variable” the metrics
denote should throw more light into the box plot. e box plots must be compared
with expected behavior.
Finally, we should mark the performance goal line on the box plot and relate
data behavior to the goal. Figure 5.1 shows the box plot of productivity metric with
the performance goal marked. is performance goal represents the project objec-
tive and not the business goal because this metric is treated as a project metric in
our example.
Meaning of metric is seen relative to the operating goal.
How data relate to performance goals is what we now learn from data. Does the
box include the performance goal line, or has the performance goal slipped away
from the box area? Or, in the worst case, has the performance goal (quantitative
target) drifted far and gone over to the whiskers?
We can now develop the various interpretations and come to some conclusions.
For instance, interpreting the productivity metric of Figure 5.1 has led to the fol-
lowing interpretations and conclusions:
Most of the results, denoted by the box and whiskers, fall short of the perfor-
mance goal or target value.
Q3
Whisker
Inner
fence
Outer
fence
Outlier
Potential
outliers
Q1
Median
Goal
–100 –50 0 50 100 150
Productivity LOC/PD
Figure 5.1 Box plot of software productivity.
78 ◾ Simple Statistical Methods for Software Engineering
Our Categories of Metrics
ere are different ways to classify metrics. We follow the five classifications of data
mentioned in Chapter 1 and work with the corresponding five types of metrics:
1. Business metrics
2. Project metrics
3. Process metrics
4. Subprocess metrics
5. Product metrics
Some of these categories could overlap depending on the definitions used in
the organization. e same metrics could be reused in a different context under
a different category. For example, reuse is generally regarded as a process metric.
However, it can be used as a product metric to evaluate product behavior, as a proj-
ect metric to understand time saved by reuse, and as a business metric to estimate
profit generated by reuse. Context influences categorization.
Business Metrics
ese are defined and deployed to implement business strategy in the organiza-
tion. e metrics are strategic in nature. A popular example of the use of business
metrics may be seen in the balanced score card framework of Kaplan et al. [3]. In
this framework, metrics are identified under four categories: finance, process, learn-
ing and growth, and customer. Kaplan promises progress through measurements.
ese metrics interact and from a system with cause–effect relationships. Deriving
business metrics from business strategy is very similar to the approach in the GQM
paradigm of Victor Basili [1]. However, instead of using a translation tree, metrics
are mapped to strategy.
Business metrics are driven by strategy and situation.
Project Metrics
Managing a software project needs some minimum metrics such as effort, sched-
ule, quality, productivity, and customer satisfaction (measured through surveys).
Project metrics are based on project requirements and customer requests. ese
are driven by project scope. e list of project metrics is extended to accommo-
date customer requirements such as reliability and maintainability. Some customers
show keen interest in knowing product quality, down to the module level. ey seek
reports on module quality and module performance during testing. Project metrics
Deriving Metrics ◾ 79
also have to satisfy information to be communicated to stake holders. For example,
resource utilization can become an important metric sought by human resources
and finance departments. Server downtime is a metric that would help the facilities
management department. Projects also measure risk by internal surveys.
In general, project metrics are requirement driven. ey are to be specified dur-
ing scope definition. Project metrics also serve as information devices and assist in
project communication.
Process Metrics
Every process presents opportunities for metrics at the input, process, and output
stages. Such a process metric is used first to understand, then to control the process,
and later to improve the same.
Use metrics to understand, control, and improve.
Software delivery is made through a chain of processes, and one or more met-
rics can be used to control and improve each process stage in the chain. e total
number of process metrics is equal to or more than the number of stages. Metric
choice is driven by process control needs. is choice of metrics is further influ-
enced by the life cycle model used. For example, in the waterfall life cycle, metrics
such as requirement stability, design complexity, code size, and test effectiveness
can be used to address control needs in each development phase.
Subprocess Metrics
A more detailed process control takes us to the subprocess. For example, a process,
such as design, can be broken down to manageable subprocesses such as design,
design review, and redesign. e metrics for controlling these subprocesses are
design effort, design review effort, and design rework effort. Subprocess metrics
Box 5.3 analogy: constRucting a Building
Software development is analogous to constructing a building in some ways.
e natural metrics in a civil construction are the size of the building, the
grade of quality of interior and materials used, and the reliability of the build-
ing against natural forces such as wind and rain. Similarly, in constructing a
software product, the basic metrics are size, complexity, and reliability. In a
construction business, routinely maintained design and construction logbooks
show these metric data. In software projects, these metrics are slowly being
accepted; a few organizations maintain these records, and many others still wait.
80 ◾ Simple Statistical Methods for Software Engineering
present a management challenge because these metrics get closer to the individuals
who fear exposure and hence refrain from sharing data. As Deming said, we need
to “drive out fear” if we want to collect subprocess metric data.
Product Metrics
Software product structure and performance are monitored through product met-
rics. For example, we can measure software code size by lines of code, structure
by function point, quality by defect density, and reliability by mean time between
failure. In the design stage, we can measure structure by function point and size by
number of design pages. In the requirement stage, we can measure size by number
of features. Product metrics can be used to monitor “product health.”
Case Study: Power of Definitions
e definition of metrics precedes metric data. Metric definitions shape our
approach to engineering before data validates our approach. e names of met-
rics with their definitions are part of the engineering and management vocabulary.
Richer vocabulary reflects richer practices. Asking how many metrics we need is
like asking how many words we need to converse effectively.
A few words are enough to exchange pleasantries,
A hundred are enough to manage simple conversations,
A thousand makes one an expert communicator, and
Many more are used by professionals.
Box 5.4 eaRthQuake
Earthquake prediction uses esoteric metrics. For example, the rise and fall
of water levels in deep wells in China are measured; elsewhere, in a scientific
approach, weak magnetic signals are monitored using sophisticated equip-
ment to measure seismic activity. Seismic vibrations are picked up along
fault lines and analyzed to predict earthquakes. All these metrics are special,
expensive, and based on geological models. Likewise, software reliability pre-
diction needs special metrics. Metric choice depends on the reliability model
selected for prediction. ese metrics are not routine but special, and they
belong to the category of “model-driven metrics.” ey have to be specially
designed and deployed, and the extra cost and effort should be approved by
business leaders.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.