Based on his experience in OO software development, Lorenz (1993) proposed eleven metrics as OO design metrics. He also provided the rules of thumb for some of the metrics, which we summarized in Table 12.1..

As the table shows, some of these metrics are guidelines for OO design and development rather than metrics in the sense of quantitative measurements. Although most of these eleven metrics are related to OO design and implementation, metric 8 is a statement of good programming practices, metric 9 is a quality indicator, and metric 11 is a metric for validating the OO development process.

With regard to average method size, a large number may indicate poor OO designs and therefore function-oriented coding. For average number of methods per class, a large number is desirable from the standpoint of code reuse because subclasses tend to inherit a larger number of methods from superclasses. However, if the number of methods per object class gets too large, extensibility will suffer. A larger number of methods per object class is also likely to complicate testing as a result to increased complexity. Too many methods in a single class, not counting inherited methods, is also a warning that too much responsibility is being placed in one type of object. There are probably other undiscovered classes. On this point, similar reasoning can be applied to instance variables—a large number of instance variables indicates that one class is doing more than it should. In other words, the design may need refinement.

Inheritance tree depth is likely to be more favorable than breadth in terms of reusability via inheritance. Deeper inheritance trees would seem to promote greater method sharing than would broad trees. On the other hand, a deep inheritance tree may be more difficult to test than a broad one and comprehensibility may be diminished. Deep class hierarchy may be the result of overzealous object creation, almost the opposite concern of having too many methods or instance variables in one class.

The pertinent question therefore is, what should the optimum value be for OO metrics such as the several just discussed? There may not be one correct answer, but the rules of thumb by Lorenz as shown in Table 12.1. are very useful. They were derived based on experiences from industry OO projects. They provide a threshold for comparison and interpretation.

In 1994 Lorenz and Kidd (1994) expanded their metrics work by publishing a suite of recommended OO metrics with multiple metrics for each of the following categories: method size, method internals, class size, class inheritance, method inheritance, class internals, and class externals. They also showed the frequency distribution of the number of classes for five projects, in the histogram form, along the values of some of the metrics. No numeric parameters of these metrics (e.g., mean or median) were provided, however.

In early 1993, IBM Object Oriented Technology Council (OOTC) (1993) published a white paper on OO metrics with recommendations to the product divisions. The list included more that thirty metrics, each with a relative importance rating of high, medium, or low. All proposed metrics by Lorenz in 1993 (Table 12.1.), with the exception of metrics 5 and 6, were in the IBM OOTC list with a high importance rating. Almost all of the OOTC metrics were included in Lorenz and Kidd’s (1994) comprehensive suite. This commonality was not a coincidence because both Lorenz and Kidd were formerly affiliated with IBM and Lorenz was formerly the technical lead of IBM’s OOTC. As one would expect, Lorenz’s OO metrics rules of thumb were the same as IBM OOTC’s. The OOTC also recommended that the average depth of hierarchy be less than 4 for C++ projects. In terms of project size, the OOTC classified projects with fewer than 200 classes as small, projects with 200 to 500 classes as medium, and projects with more than 500 classes as large.

Table 12.2 shows selected metrics for six OO projects developed at the IBM Rochester software development laboratory. Project A was for the lower layer of a large operating system that interacts with hardware microcode; Project B was the development of an operating system itself; Project C was for the software that drives the input and output (IO) devices of a computer system; Project D was for a Visualage application; Project E was for a software for a development environment, which was a joint project with an external alliance; and Project F was for a software that provides graphical operations for a subsystem of an operating system. Based on OOTC’s project size categorization, Projects A, B, and C were very large projects, Projects E and F were medium-sized projects, and Project D was a small project.

Compared with the rules of thumb per Lorenz (1993) and IBM OOTC (1993), Project E had a much higher average number of methods per class, a larger class in terms of LOC, and a larger maximum depth of inheritance tree. Project E was a joint project with an external alliance and when code drops were delivered, acceptance testing was conducted by IBM. Our defect tracking during acceptance testing did show a high defect volume and a significantly higher defect rate, even when compared to other projects that were developed in procedural programming. This supports the observation that a deep inheritance tree may be more difficult to test than a broad one and comprehensibility may be diminished, thereby allowing more opportunities for error injection.

The metric values for the other projects all fell below the rule-of-thumb thresholds. The average methods per class for projects A, B, and C were far below the threshold of 20, with project B’s value especially low. A smaller number of methods per class may mean larger overheads in class interfaces and a negative impact on the software’s performance. Not coincidentally, all three projects were not initially meeting their performance targets, and had to undergo significant performance tuning before the products were ready to ship. The performance challenges of these three projects apparently could not be entirely attributed to this aspect of the class design because there were other known factors, but the data demonstrated a good correlation. Indeed, our experience is that performance is a major concern that needs early action for most OO projects. The positive lesson learned from the performance tuning work of these projects is that performance tuning and improvement are easier in OO development than in procedural programming.

In 1994 Chidamber and Kemerer proposed six OO design and complexity metrics, which later became the commonly referred to CK metrics suite:

Chidamber and Kemerer (1994) applied these six metrics in an empirical study of two companies, one using C++ and one using Smalltalk. Site A, a software vendor, provided data on 634 classes from two C++ libraries. Site B, a semiconductor manufacturer, provided data on 1,459 Smalltalk classes. The summary statistics are shown in Table 12.3.

The median weighted methods per class (WMC) for both sites were well below the threshold value for the average number of methods (20) as discussed earlier. The DIT maximums exceeded the threshold of 6, but the medians seemed low, especially for the C++ site. The classes for both sites had low NOCs—with medians equal to zero, and 73% of site A and 68% of site B had zero children. Indeed the low values of DIT and NOC led the authors to the observation that the designers might not be taking advantage of reuse of methods through inheritance. Striking differences in CBOs and RFCs were shown between the C++ and Smalltalk sites, with the median values for the Smalltalk site much higher. The contrast reflects the differences in the lan-guages with regard to OO implementation. Smalltalk has a higher emphasis on pure OO message passing and a stronger adherence to object-oriented principles (Henderson-Sellers, 1995). Last, the distribution of lack of cohesion on methods was very different for the two sites. Overall, this empirical study shows the feasibility of collecting metrics in realistic environments and it highlights the lack of use of inheritance. The authors also suggested that the distribution of the metric values be used for the identification of design outliers (i.e., classes with extreme values).

To evaluate whether the CK metrics are useful for predicting the probability of detecting faulty classes, Basili and colleagues (1996) designed and conducted an empirical study over four months at the University of Maryland. The study participants were the students of a C++ OO software analysis and design class. The study involved 8 student teams and 180 OO classes. The independent variables were the six CK metrics and the independent variables were the faulty classes and number of faults detected during testing. The LCOM metric was operationalized as the number of pairs of member functions without shared instance variables, minus the number of pairs of member functions with shared instance variables. When the above subtraction is negative, the metric was set to zero. The hypotheses linked high values of the CK metrics to higher probability of faulty classes. Key findings of the study are as follows:

This validation study provides positive confirmation of the value of the CK metrics. The authors, however, caution that several factors may limit the generalizability of results. These factors include: small project sizes, limited conceptual complexity, and student participants.

In 1997 Chidamber, Darcy, and Kemerer (1997) applied the CK metrics suite to three financial application software programs and assessed the usefulness of the metrics from a managerial perspective. The three software systems were all developed by one company. They are used by financial traders to assist them in buying, selling, recording, and analysis of various financial instruments such as stocks, bonds, options, derivatives, and foreign exchange positions. The summary statistics of the CK metrics of these application software are in Table 12.4.

One of the first observed results was the generally small values for the depth of inheritance tree (DIT) and number of children (NOC) metrics in all three systems, indicating that developers were not taking advantage of the inheritance reuse feature of the OO design. This result is consistent with the earlier findings of an empirical study by two of the authors on two separate software systems (Chidamber and Kemerer, 1994). Second, the authors found that three of the metrics, weighted methods per class (WMC), response for a class (RFC), and coupling between classes (CBO) were highly correlated, with correlation coefficients above the .85 level. In statistical interpretation, this implies that for the three software systems in the study, all three metrics were measuring something similar. This finding was in stark contrast to the findings by the validation study by Basilli and colleagues (1995), in which all six CK metrics were found to be relatively independent. This multi-collinearity versus independence among several of the CK metrics apparently needs more empirical studies to clarify.

We noted that the metrics values from these three systems are more dissimilar than similar, especially with regard to the maximum values. This is also true when including the two empirical data sets in Table 12.3. Therefore, it seems that many more empirical studies need to be accumulated before preferable threshold values of the CK metrics can be determined. The authors also made the observation that the threshold values of these metrics cannot be determined a priori and should be derived and used locally from each of the data sets. They decided to use the “80-20” principle in the sense of using the 80th percentile and 20th percentile of the distributions to determine the cutoff points for a “high” or “low” value for a metric. The authors also recommended that the values reported in their study not be accepted as rules, but rather practitioners should analyze local data and set thresholds appropriately.

The authors’ major objective was to explore the effects of the CK metrics on managerial variables such as productivity, effort to make classes reusable, and design effort. Each managerial variable was evaluated using data from a different project. Productivity was defined as size divided by the number of hours required (lines of code per person hour), and was assessed using data from software A. Assessment of the effort to make classes reusable was based on data from software B. Some classes in software B were reused on another project and the rework effort was recorded and measured in the number of hours spent by the next project staff to modify each class for use in the next project. Assessment on design effort was based on data from software C. Design effort was defined as the amount of time spent in hours to specify the high-level design of each class. Multivariate statistical techniques were employed with class as the unit of analysis and with each managerial variable as the independent variable. The CK metrics were the independent variables; other relevant variables, such as size and specific developers who had superior performance, were included in the models to serve as control variables so that the net effect of the CK metrics could be estimated. The findings indicated that of the six CK metrics, high levels of CBOs (coupling between object classes) and LCOMs (lack of cohesion on methods) were associated with lower productivity, higher effort to make classes reusable, and greater design effort. In other words, high values of CBO and LCOM were not good with regard to the managerial variables. Specifically, the final regression equation for the productivity evaluation was as follows:

The equation indicates that controlling for the size of the classes and the effect of a star performer (STAFF_4), the productivity for classes with high CBO (coupling between object classes) and LCOM (lack of cohesion on methods) values was much lower (again, the authors used the 80th percentile as the cutoff point to define high values). Because productivity was defined as lines of code per person hour, the regression equation indicates that the productivity was 76.57 lines of code per hour lower (than other classes) for classes with high CBO values, and 33.96 lines of code per hour lower for classes with high LCOM values. The effects were very significant! As a side note, it is interesting to note that the productivity of the classes developed by the star performer (STAFF_4) was 48.11 lines of code per hour higher!

This finding is significant because it reflects the strength of the underlying concepts of coupling and cohesion. In practical use, the metrics can be used to flag out-lying classes for special attention.

Rosenberg, Stapko, and Gallo (1999) discuss the metrics used for OO projects at the NASA Software Assurance Technology Center (SATC). They recommend the six CK metrics plus three traditional metrics, namely, cyclomatic complexity, lines of code, and comment percentage based on SATC experience. The authors also used these metrics to flag classes with potential problems. Any class that met at least two of the following criteria was flagged for further investigation:

Considerable research and discussions on OO metrics have taken place in recent years, for example, Li and Henry (1993), Henderson-Sellers (1996), De Champeaux (1997), Briand (1999), Kemerer (1999), Card and Scalzo (1999), and Babsiya and Davis (2002). With regard to the direction of OO metrics research, there seems to be agreement that it is far more important to focus on empirical validation (or refuta-tion) of the proposed metrics than to propose new ones, and on their relationships with managerial variables such as productivity, quality, and project management.

To transition from structural design and procedural programming to OO design and programming, it is crucial to allow time for the learning curve. Formal classroom education is needed. A curriculum customized to the organization or specific projects and timely delivery are major considerations in a training program. The most effective curriculum spreads out class time with on-the-job training. Consultants or trainers should be industry experts or experienced in-house experts with full-time training jobs, so that classroom education as well as consulting and trouble shooting when the project is under way can be provided. The education should include the entire design and development life cycle (requirements, analysis, design, coding, and testing) and not focus only on language and programming. Based on experience, it takes six months to complete OO skills training based on the model that classroom education is spread out with on-the-job training: three months in calendar time to complete initial training followed by three months of advanced training. After training, it takes three months to become capable, that is, to acquire the working knowledge to do OO design and development. It takes another nine to twelve months to become proficient in OO technology, provided that the developers have been working on a project.

Like any software projects, domain-specific knowledge and skills and software engineering skills are equally important. For OO software engineering, if we dichotomize domain knowledge and OO skills and cross-tabulate the two variables, we get the four skills categories shown in Figure 12.6.

Figure 12.6. Domain Knowledge and OO Skills Typology

For developers who are not experienced in OO skills and domain knowledge (e.g., a banking application software, or an operating system architecture and design), the progression in skills would be from category I to II to IV. For experienced developers with rich domain-specific knowledge who are transitioning to OO technology, the transition path would be from category III to IV. For an OO development project, at any phase in the development process, there should be OO experts (category IV skills) leading the projects and providing on-the-job guidance to those who are less experienced. The composition of skills categories and the percentage of OO experts can vary across the development cycle. The percentage of OO skills over the phases of the development cycle can be distributed as shown in Figure 12.7 so that the project has enough OO experts to enhance its success, and at the same time has the capacity for skills development. In the figure, the line curve approximates the staffing curve of a project, and the phases are characterized by different distribution of the analysis, design, code, and test (ADCT) activities.

Figure 12.7. Skill Requirements for the Development Cycle

A consistent process architecture in an organization is important so each new project does not require reinvention of the wheel. Consistency is also the basis for systematic improvements. If at all possible, use a mature tool set and a stable development environment, and have peer support for the language that is being used. Many good tools exist in the industry. But the robustness of the tools and the integration of tools to provide cohesive development environments are even more important. Tools and development environment are as important as the languages and specific OO methods to be used for the project. Developing your own tools should be considered only as a last resort. At the same time, it is very advantageous if the development team knows its compiler well because changes may be needed for performance tuning.

Good project management is ever more important due to the iterative nature of the OO development process. We recommend monthly goals or mini-milestones be set with the project schedule, and explicit and frequent checkpoints be conducted. The use of metrics to show the status and trends of the project progress, such as those shown in Figures 12.3 through 12.5, enhances the focus of the team. As illustrated in the example illustrated with these figures, metrics are applicable to large projects as well as small ones. The measurements should be decided at the outset and the team should ensure that everyone knows the measurements and that measurements are collected regularly. Status reporting is another area that must be watched because developers might regard status reporting as dead time. It is not rare that teams disclose the information that they are going to fail to meet a target only when they have actually failed.

Inevitably, some large software projects involve multiteam and multilocation development. Our experience suggests that, because of the iterative nature of the OO process, such projects require more coordination and better communication than projects using the conventional approach.

Directly related to the iterative nature of the OO development process are two major items on the top of the project management list: (1) beware of analysis and design without code—the so called analysis/paralysis phenomenon, and (2) beware of prototypes that never end. Prototypes play an important role in OO development and are often used in building a fully functional user interface or in showing a proof of feasibility of the project. Seeing is believing; prototypes usually command high visibility from customers, project managers, and particularly executives of the organization. However, there is a big gap between the completion of prototypes and the completion of the design and development of the project. To fill the gap, rigorous work on the overall architecture, design (internal as well as external), scaling, and implementation needs to happen. Technical leaders need to provide the leadership to bridge the gap and project management needs to plan and drive the necessary activities and milestones after successful prototypes were achieved. We have seen OO projects that kept on prototyping and iterating, that mistook the success of prototypes as success of the project, and that significantly underestimated the design and development work after prototypes and therefore failed to meet product delivery targets. Warning signs include hearing conversation among developers like the following: “What have you been up to in the past several months?” “I have been working on our prototype project.”

Good reusable classes are hard to build. To encourage reuse, some projects purchased libraries of reusable classes, some projects created internal programs (e.g., a reuse council), and some projects took both approaches. But they resulted in only a slight increase in reuse, and there were few submissions to the reuse program. This was a clear contrast to the original anticipation associated with the OO technology and process. This low level of reuse is consistent with the findings in the literature (Kemerer, 1999) with regard to inheritance and reuse.

Our lesson learned is that OO technology is not automatically equal to code reuse. To have tangible results, the team has to design for it, and must continually work at it. Class reuse issues may not even be able to be adequately addressed at the project level. From the perspective of product development, there is an inherent conflict of interest between development cycle time and the building of reusable classes. It is more effective to have a reuse strategy in place at the organization level so that issues related to the management system, interproject coordination, libraries for reusable classes, possible incentive program, and so on, can be dealt with on a long-term basis.

Performance is a major concern in most, if not all, OO projects during development, especially new projects. It is not uncommon that initial measures of performance cause serious distress. We did see projects fail due to performance problems. Performance problems can be attributed to several factors related to experience and technology:

With regard to the compiler factor, its contribution to performance problems should be lesser now than in the early days of OO technology. Although the compiler technology has improved, it remains a concern to OO development experts. A recent assessment of an OO project that had to be rewritten due to performance problems confirms that compiler optimization (or the lack thereof) remains a critical factor.

On the positive side, our experience indicates that performance tuning in OO structure is not too difficult to achieve, especially if it is done early. In most cases, performance problems were overcome when the project was complete. The lesson with regard to performance: It is a concern to be taken seriously, and early performance measurement and tuning are crucial.

Assessments by developers were that in OO development, bugs are easier to find and fix. Our data also indicate that more often than not, defect rates for OO projects are indeed lower during the testing phases as well as when the products are in the field. The data on OO quality for a few projects that were discussed earlier in this chapter do support these observations. Type checking abilities of OO compilers, the practice of object coherence checking, the class structure in OO designs, the focus on analysis and design in the OO development process, and design tools are among the many positive factors for quality.