Approaches to Development Metrics

Development defies measurement in many ways. Unlike staffing, development is a virtual kaleidoscope of activities, only a portion of which are under the control of the organization. Development ranges from the informal and nearly impossible to measure, when one employee teaches another how to do something, to the formal and easily measured activity of employees attending formal training. It would be easy to throw up our hands in frustration, claiming it can’t be done. But organizations demand accountability, so it is incumbent upon us to think creatively to produce metrics that advance accountability for development, even if they are still imperfect.

There has been a variety of attempts to create development metrics. In this section, we review six different approaches to measuring development. For each approach, we discuss the metrics, present examples, and identify the shortcomings in each that have led us to create new metrics for widespread use.

The American Society for Training and Development Approach. Each year the American Society of Training and Development (ASTD) prepares a state of the industry report that provides a comprehensive overview of employer-provided training in the United States. This report provides organizations with important benchmarking information for training, learning, and performance improvement processes, practices, and services. In short, it’s a snapshot of investments and expenditures made in training across organizations.

ASTD Metrics Analysis. Unquestionably, these key figures (such as total training expenditures as percent of payroll, percent of training-eligible employees trained, training-eligible employees to trainer ratio, and so on) provide interesting training-related information. Another significant benefit of these measures is that they are based on readily available data. For instance, the calculation of total training expenditures per training-eligible employee is quick and easy since both figures are usually readily available in most organizations. It is difficult to imagine a scenario in which an extensive data collection effort would be required to obtain the figures necessary to make any calculation in the ASTD metrics.

However, in many ways the shortcomings of these measures mirror the problems with cost-per-hire, the traditional staffing metric discussed in the last chapter. Consider training as a percent of payroll, which has traditionally been used as the key metric. What does it really mean? Is any manager using it to decide how much training to offer employees? Is any organization that makes strategic training decisions using it? While it is an interesting benchmark, it does not seem to be one that has led to increases in training budgets within organizations, even though training investments have increased.

Furthermore, because these metrics focus solely on training, their scope is entirely too limited to be used as effective development metrics. Other developmental programs and initiatives (such as OJT, coaching, mentoring, and so on) are not included, thereby leaving a tremendous void in their usefulness as developmental metrics. As noted earlier, development, unlike training, is not limited to structured learning activities designed to help employees fulfill job duties. Instead, it extends past training to include short- and long-term activities. Unless an organization has an extraordinarily good human resource accounting system, these activities are not likely to be included in training expenditures. The general trend in organizations today is to use more nontraining initiatives, making these metrics even less useful. By omitting or overlooking developmental efforts that extend beyond training initiatives, the ASTD metrics are incomplete and inadequate for our use.

The Human Resource Development Evaluation Approach. Training and human resource development professionals have taken a completely different approach to measuring development. Unlike the organization-level metrics employed in the ASTD approach, they have focused on intervention or program-level measures through what is typically called “evaluation.” Under this approach, individual development interventions or programs are evaluated, and the results are aggregated to provide organization-level metrics.

There are two basic problems with this approach:

Therefore, the metrics approach presents an alternative to accountability. It is an easier approach and one more likely to be used. Program-level results assessment is very useful for diagnosis of programs, and while we support program-level results assessment, results have clearly shown that development of it does not fit our purpose (Swanson 1998). The persistent low levels of training evaluation, particularly at the more sophisticated levels three and four, raise serious questions about the state of training evaluation research as well as whether currently used models will enable trainers to achieve higher levels of program evaluation. Simply stated, in 40 years of promoting, its use has not changed the overall picture; something else must be needed.

The Financial Approach. Traditionally, Human Resources has not looked to Finance and Accounting to create metrics. While it will become clear that metrics other than financial ones will be needed, this separation between Human Resources and Finance is unnecessary and counterproductive. Thus, we will consider some basic financial metrics to inform our new development metrics.

The financial approach represents selected financial metrics related to development that have arisen from human capital economics, utility analysis from industrial-organizational psychologists, intellectual capital, and financial analysis. The financial approach has been to use existing financial measures to place value on human capital.

Intellectual capital theory posits that some employees are more productive than others due in large part to their acquired knowledge, skills, and abilities. The presumption is that returns from human capital are represented by the difference between the worth of a firm’s assets and the value placed on it by the stock market.

An example of this approach is the Human Capital Return Metric. Intellectual capital theory has attempted to use financial measures to determine the return from human capital. One key metric for human capital is the following:

Human Capital Return = Market Value – Book Value

From this perspective all returns over the book value of the firm are attributable in returns from human capital development. However, book value is often too conservative, so another approach is to use Tobin’s Q, which is:

This ratio controls different depreciation policies that affect book value. Values greater than one indicate returns from intellectual capital.

Intellectual capital theory is fundamental to any attempt to create development metrics; however, the use of market value to calculate human capital returns is problematic except in the long run. We need only look to the stock market in the late 1990s (particularly Internet stocks) to see how market valuations can become disconnected from real firm performance. Thus, linking development metrics to stock market valuations could create tremendous short-term volatility in the metric, rendering it unusable.

Human Resources/Performance Research Approach. There has been a flurry of research in recent years linking Human Resource best practices to organizational performance. Research suggests that human resources do make a difference. There are several core premises underlying the performance approach to Human Resources:

• Performance-based Human Resources must enhance current performance and build capacity for future performance.

• Training and other development initiatives cannot be separated from other parts of the performance system.

• Effective performance systems are rewarding both on an individual level and an organizational level.

Given these assumptions, it is easy to understand why the interest in human resource and performance research spans many disciplines from psychology to sociology to HR management and many others. Like the ASTD approach, the human resources and performance research approach is typically an organizational-level approach using broad metrics of Human Resources practices.

The direct correlation of these metrics with performance strengthens the case for creating metrics to measure development. These studies provide clear evidence that development (and other Human Resource activities) is related to performance.

However, while some measures of Human Resource activity have worked, most rely on information that is not readily available. Most use special surveys, some of which are labor intensive. In addition, as a practical matter, these metrics are generally not easy to attain, and are therefore not considered to be optimal or practical as our development metrics.

Human Resources Metrics Approach. We are certainly not the first to consider that new sets of organization-level metrics are needed. We will call this the Human Resource metrics approach, which includes work by Fitz-enz, Ulrich, Zenger, Smallwood, and Kaplan and Norton. In general, these researchers have adopted a similar approach to ours. That is, they have attempted to define key organization-level indicators that provide a picture of development effectiveness across a broad sample of companies. These metrics examine the following areas:

• Lagging indicators.

• Leading indicators.

• Collective assessment and quantitative collection measures.

• Employee competence.

• Learning and growth perspectives in terms of employee capabilities.

These metrics, without a doubt, are most directly associated with our goal of identifying or establishing development metrics. They also clearly demonstrate the value of organization-level metrics used in a balanced scorecard approach to HR accountability. Together they provide a useful starting point. However, they have enough flaws that we are reluctant to adopt any of them entirely. There are two primary flaws: