Following code churn, we analyzed code complexity. That is, how does the complexity of a piece of code influence the ability to predict failures? Complexity can be quantified using several metrics, more suited toward object-oriented and non-object-oriented metrics ranging from the more traditional metrics such as fan-in and fan-out to the more recent CK metrics [Chidamber and Kemerer 1994]. The CK metric suite consists of six metrics (designed primarily as object-oriented design measures): weighted methods per class (WMC), coupling between objects (CBO), depth of inheritance (DIT), number of children (NOC), response for a class (RFC), and lack of cohesion among methods (LCOM). The typical object-oriented and non-object-oriented complexity metrics used at Microsoft based on prior published metrics (for example, [Chidamber and Kemerer 1994]) are:
- Lines
Executable noncommented lines of code.
- Cyclomatic complexity
The Cyclomatic complexity metric [McCabe 1976] measures the number of linearly independent paths through a program unit.
- Fan-in
Fan-in is the number of other functions calling a given function in a module.
- Fan-out
Fan-out is the number of other functions being called from a given function in a module.
- Methods
Number of methods in a class, including public, private, and protected methods.
- Inheritance depth
Inheritance depth is the maximum depth of inheritance for a given class.
- Coupling
This signifies coupling to other classes through (a) class member variables, (b) function parameters, and (c) classes defined locally in class member function bodies, as well as (d) coupling through immediate base classes and (e) coupling through return type.
- Subclasses
This metric indicates the number of classes directly inheriting from a given parent class in a module.
The overall efficacy of using these metrics for predicting failures is with a precision of 79.3% and a recall of 66% in Windows Vista. The measures indicate that though complexity measures are fair predictors of code quality, they are not as good as code churn. In related work, Khoshgoftaar et al. studied two consecutive releases of a large legacy system (containing over 38,000 procedures in 171 modules) for telecommunications and identified fault-prone modules based on 16 static software product metrics [Khoshgoftaar et al. 1996]. Their model, when used on the second release, showed an overall misclassification rate of 21.0%. El Emam et al. studied the effect of class size on fault-proneness by using a large telecommunications application [El Emam et al. 2001]. Class size was found to confound the effect of all the metrics on fault-proneness. The CK metrics have also been investigated in the context of fault-proneness. Basili et al. studied the fault-proneness in software programs using eight student projects [Basili et al. 1996]. They observed that the WMC, CBO, DIT, NOC, and RFC were correlated with defects, whereas the LCOM was not correlated with defects. Briand et al. performed an industrial case study and observed the CBO, RFC, and LCOM to be associated with the fault-proneness of a class [Briand et al. 1999]. Tang et al. studied three real-time systems for testing and maintenance defects [Tang et al. 1999]. Higher WMC and RFC were found to be associated with fault-proneness. For further reading, refer to [Nagappan et al. 2006a], [Briand et al. 1999], and [Bhat and Nagappan 2006].