Project management and process improvement through software metrics
Goals, questions, and metrics
Many organizations waste time and money by measuring more things than they need to and by
measuring the wrong things. It is expensive to design and develop data collection mechanisms,
collect and store data, extract data, generate metrics reports, analysis the reports, and, finally,
understand what the information means.
Goal/Question/Metric
The Goal-Question-Metric (GQM)
paradigm, illustrated in Figure 2, Goal 1 Goal 2
developed by Victor Basili and
associates at the University of Maryland,
provides an excellent mechanism for
defining a goal-based measurement
program.
Set Goals
Determine how you want to improve your projects and products. Your goal, for example, might be
to reduce the number of defects in the soflware, thus minimizing cost and maximizing customer
satisfaction. Or you may have specific goals, such as evaluating the effectiveness of a new process
or determining if a product is ready for release.
Ask Questions
Determine what questions you need to ask in order to meet your goals. A question might be,
"What kind of defects are costing us the most?"
Establish Metrics
Set up metrics that will answer your questions. You might start collecting data on defect types,
creation times, detection times, costs to detect, and costs to correct.
For example, if our high-level goal is to "produce defect-free software," this could result in the
following questions:
-
Are our defect detection techniques effective?
- Is the software adequately tested?
- How many defects are still undetected?
- Are all known defects corrected before release?
The forth question-''Are all known defects corrected before release? '
Number of defects detected.
Percentage of known defects corrected.
Percentage of corrections tested.
Percentage of test cases regression tested.
Designing a metric
The metrics objective statement defines the requirements for the metric. As in any other software
project, the requirements must be translated into a design. This section outlines a step-by-step
process for defining the design of a software metric.
Step 1 - Clear Definitions
The first step in designing a metric is to agree to a standard definition for the entities and their
attributes being measured. When we use terms such as "defect," "problem report," "size," and even
"project," other people will interpret these words in their own context with meanings that may
differ from our intended definition. These interpretation differences increase when more
ambiguous terms such as "quality, " "maintainability," and "user-friendliness" are used.
Step 2 - Define the Model
The next step is to derive a model for the metric. In simple terms, the model defines how we are
going to calculate the metric. Most modeling includes an element of simplification. Remember
that the model can always be modified to include additional levels of detail in the future. Questions
to ask when defining a metrics model include:-
Does the model provide more information than we have now?
- Is the information of practical benefit?
- Does it tell us what we want to know?
To illustrate the selection of a model, let's consider a metric for the duration of unplanned system
outages. If we are evaluating a software system installed at a single site, a simple model, such as
minutes of outage per calendar month, maybe sufficient. If our objective is to compare different
software releases installed at varying numbers of sites, we might select a model such as minutes of
outage per 100 operation months. Or, if we wanted to focus in on the impact to our customers, we
might select minutes of outage per site per year.
