Vague, ambiguous metric definitions allow every practitioner to interpret them differently. One person counts an unnecessary software feature as a defect of type "Extra," whereas someone else does not. Time spent fixing a coding error discovered through testing is classified as test effort by one person, construction effort by another, and rework by a third. Trends in measures tracked over time might show erratic behavior because individuals are not collecting, reporting, or charting their results in the same way.

You’ll have a clue that your metrics definitions are inadequate if participants are frequently puzzled about what exactly they are being expected to measure. If you keep hearing questions such as, "Do I count unpaid overtime in the total work effort?" you may be falling into this trap.

A complete and consistent set of definitions for the things you are trying to measure is essential if you wish to combine data from several individuals or projects. For example, the definition of a line of code is by no means standard even for a single programming language: Physical or logical statements? Does formatting matter? Do comments count? How about data declarations? Standardize on the tools for collecting metrics and automate the measurement process where possible. Use standard calibration files to make sure all participants have configured their measurement tools correctly.

Those designing the metrics program must create a precise definition for each base measure being collected, as well as for derived measures computed from combinations of these base measures. One of my metrics planning teams found this to be considerably more difficult than we expected. Plan to spend considerable time agreeing on definitions, documenting them as clearly as possible, and writing procedures to assist practitioners with collecting the base measures easily and accurately.