Collect Data Centrally

The first building block for effective measurement is to collect data centrally. All assessment results should be gathered in one place. Interview ratings, psychometric scores, competency ratings: all need to be collated from throughout the business. Wherever measurement data exist, they need to be drawn in. It may be possible to use an HR information technology system to store the data, but at the very least, there needs to be a large, central spreadsheet containing all results and a secure place to keep all reports. The centrality of the database is critical too, so that the data can be shared. In businesses with both recruitment and learning teams, for example, it should sit outside both so it is not viewed as belonging to one, but not the other.

Without centrally collecting talent data, businesses cannot build up a picture of the talent across the organization. Instead, all they will have are fragments of information relating only to individuals or teams. Not collecting measurement data is like buying a sports car and then leaving it on the driveway or using it only to drive the kids to school. You will only ever enjoy a small margin of the value that the car can provide.

Even when firms do not use measurement methods much, they should still collect the data so that they can follow trends over time. In some firms, for example, measurement is limited to recruitment interviews, and the only outputs are a whether-to-hire rating and perhaps competency ratings. These may not seem like much, but they can be extremely useful and ought to be collected. Data are only as useful as the insights they give, and the insights they give are vastly increased when they are connected with other data.

For example, knowing the average competency ratings of newly hired employees can be useful. Yet if you also know the performance scores of the newly hired one year after they have joined the business, you can see which competencies are most predictive of initial success. If you know who is still employed three years later, you can work out which factors are most predictive of retention. And knowing who is later promoted can provide insight into the types of talent valued in your business and the qualities predictive of longer-term success.

Moreover, because companies need not only to gather talent data, but also to join those data with other information, they should never be tempted to let a vendor take care of data collection for them. Businesses must own and control their measurement data, because only they can connect these results to other talent data. Right from the start, they need to record other relevant talent data alongside the measurement results—things like performance scores, promotion dates, business unit, location, and key demographic data. And it is critical to do this from the very beginning because the fragmented nature of many larger companies' systems and processes can make gathering this information extremely challenging in retrospect.

We look in more detail at what organizations can do with measurement data in the next chapter. For now, the key point is that if measurement results are to be useful for more than merely informing individual people decisions, data need to be collected centrally. They are valuable resources and should be managed as such.

Use Common Data Points

Just collecting talent measurement data centrally is not enough, however. For this information to be meaningful and useful, you need to have common data points. For example, if you measure one person's intelligence and another's personality, bringing the two pieces of information together will not tell you much. But if you know the personalities of both people, then you can compare them. And if you know this for enough people, you can compare individuals to the average profiles of a group or the qualities of different groups, such as the leadership teams of different business units. It is therefore critical that as far as possible, you know the same information about different employees. Without this, any measurement database will be of little value in what it can tell you, and talent analytics will simply not be possible.

One significant step in this direction can be to ensure that if you use intelligence or personality tests in your business, you use the same tests and not different ones from different vendors. This is important because comparing the scores from two different tests can be very difficult and is often impossible. Using the same tests is not always easy to achieve, though, since business or HR leaders often have their own favorite tools. And for large multinationals, it may not be completely possible: they may find, for instance, that the intelligence test that most suits the majority of the business is not available in some locations. Yet as far as possible, the guiding principle ought to be that the tools and tests you use should be standardized.

In many businesses, the main tools used to measure talent are competency frameworks. These describe the skills, behaviors, and attributes that individuals are thought to need to be successful. They can exist for single job families, but can also be used to define the capabilities required in particular groups of people across whole business units or organizations. For example, many businesses have a set of leadership competencies that are used for all of their senior leaders or technical competencies specific to particular functions. As a result, competency frameworks provide an excellent opportunity for ensuring there are common data points.

There is a tension here, however. On the one hand, the fewer frameworks your business has, the more able it will be to collate and compare competency ratings from different parts of the business. Yet on the other hand, as we saw in chapter 3, when assessments take into account the specific circumstances and needs of a business unit or team, then those assessments are likely to be more accurate in predicting individuals' success in those environments. So the more specific and personal that a competency framework is to your company or business unit, the more effective it is likely to be for measuring talent. And if we follow this line of reasoning, then larger businesses may well be better off with lots of different frameworks, each specific to a particular part of the organization.

So although a single common framework is probably appropriate for small firms, a compromise is required for large or diversified businesses. They need both business-specific frameworks that enable an accurate evaluation of the fit between individuals and business needs and common data points in order to be able to use the measurement data for more than just informing individual people decisions.

The solution here is compromise. The guiding principle should always be to have as few frameworks as possible, so that there are some common data points between people of different levels and in different parts of the business. Yet this still leaves room for competencies specific to particular populations. One approach that we have seen work well is to have both generic and business unit–specific behavioral competencies, in addition to technical ones. This allows both common data points and tailored, business-specific assessments. For example, a company could have a framework that consists of:

• Technical competencies specific to particular functions or job families. In small firms, this could be a single competency called “technical knowledge.” In organizations with significantly sized functions, there could be a collection of competencies for each technical area.
• Two or three behavioral competencies specific to each of its business units that describe what is required to succeed in each area.
• Three or four generic behavioral competencies against which everyone is evaluated. If some abilities are specific to the leadership populations, then an additional one or two can be added for this group.

As an example of this approach, consider a large, global energy firm we worked with. It had a competency framework that it used across the whole firm to collect common data points. One business unit, though, decided to supplement this with three additional competencies. It believed that these more accurately described the necessary qualities to succeed in its particular business area. Two years later, it was proved correct. The three unit-specific competencies more accurately predicted performance than the wider firm's generic framework.¹

One final point to note here is that not just any competencies will do: they need to be suitable for measurement (and not all of them are). Developing effective competency frameworks is a subject worthy of a book in its own right. Because our focus here is the need for common data points rather than competencies, we will not dwell further on their construction at this time. However, in the appendix at the back of this book, we address the issue of how you can tell whether a framework is suitable for measuring talent.

Collecting common data points is thus an essential foundation for obtaining meaningful data. Like collecting data centrally, it is an absolute must for organizations hoping to build up a picture of the talent across the business.

Check the Impact of Measures

The third must-do action for businesses in making measurement effective is to check whether the methods and tools they use work. All too often, companies see that a tool has a certain validity and then seem to take it for granted that it will have the desired impact. In this respect, when it comes to measurement, organizations appear to be running on faith. A recent survey showed that 79 percent believe that measurement adds value to their organization, but only 23 percent actually check that it does.²

Why is checking impact so important? One obvious reason is that spending money on a service without having any idea if it works is plainly bad business. A less obvious one is that measurement methods are not like most other products that you buy. Despite what some vendors may try to tell you, most of the time most tools do not reach you as finished products. If you are using psychometric tests with a limited number of people, then there is little, if any, finishing to do, and using them straight off the shelf is fine. But most methods require some extra tuning, especially if you are going to be using them a lot (for example, with over one hundred people). Norm groups need to be built, exercises adjusted, and tools tailored to measure fit with a particular organization, business unit, or role. None of this can happen without some feedback about whether the measure is in fact working.

We live in a world where most of the products and services we buy and use work perfectly the first time. We expect this, even demand it. But talent measures are precision instruments, and they require optimizing. You should not expect them to work perfectly straightaway. Indeed, when you first check their efficacy, you should expect to find room for improvement.

Nancy Tippins, a past president of the American Psychological Association's organizational psychology division, has spoken passionately about this. As she puts it, “Businesses often buy a measurement product and then do not do anything else with it. They expect it to work straight out of the box. To get the most from these products, though, they need to continually hone and optimize them.”³ Evaluating impact is important not only for checking if measures work, but also for making them work.

Some companies require vendors to check these issues for them. But no matter how positive your relationship with a vendor is, asking it to evaluate its own tools is not advisable. There is just too much vested interest. By all means, ask a vendor to help share the work involved, but you must own and lead your own process.

Moreover, we are not simply talking about checking and honing the efficacy of a vendor's products. Evaluating impact is also vital to help managers improve their people decisions. For example, one business we work with always follows up the “failure” of new employees with a review meeting involving the hiring manager or recruitment lead. The meeting usually lasts no more than thirty minutes. During it, the information available at the time of hiring is reviewed to see if any potential signs of failure were missed. The idea is not to lay blame, but to have a genuinely curious inquiry with the aim of helping people improve their selection skills.

So how can businesses go about evaluating impact? There are two main ways. There are simple validity studies, which check whether ratings or results predict subsequent performance, and there are impact studies. For promotion decisions, an example of an impact study would be following up to check success rates among those promoted. For recruitment decisions, meanwhile, quality of hire metrics can be used. These include a variety of indicators. The trick is not to track them all, but to select and focus on just a few. Monitoring these indicators need not be complicated, either. In fact if you have the first two foundations in place—centrally collected data and common data points—then it can be quick and easy to check the data regularly. The list of possibilities is long and includes the following:

• Retention figures
• Performance ratings
• Hiring manager's satisfaction with new employees
• New employees' reactions to the measurement process
• New employees' end-of-year bonus allocation
• Comparison of competency ratings at hiring interview and at first annual appraisal
• Whether a new employee is tagged as high potential within an agreed time period
• The number of legal challenges to a selection process

So our third foundation is the need to check whether measures work and their impact on actual people decisions—not as some kind of after-the-fact evaluation but as an integral part of making measurement work.

Ensure Proper Use

The first three foundations are all fairly simple. They do not take significant time or resources. All they really need is a little will. The final foundation can be a bit tougher. It is the need to ensure that the tools and methods of measurement are used properly. You can have the best tests in the world, but if you do not use them properly, they are not going to work as well as they should.

There are two issues here. The first is making sure that tools are used for proper and appropriate purposes. For example, some types of personality psychometrics—called ipsative tests—are suitable only for development processes, not selection (see the appendix). Some types of tools do not work with particular populations. English-language intelligence tests, for instance, should be used only with people for whom English is their mother tongue. And some types of tools should probably never be used, such as Facebook content and credit scores.

The second issue is making sure that the outputs of mea­surement processes—the ratings, results, and reports—are used properly. For example, you can make sure that managers do not hire only one personality type or that hiring decisions do indeed take measurement results into account. We have seen too many businesses that have put great effort into identifying the best tools to use, only to leave it to line managers to decide how they want to use them. And inevitably this leads to mixed or poor use of the outputs.

These two issues are usually easier for smaller companies than larger ones. In decentralized businesses, they can be particularly challenging. Nonetheless, organizations need to do something, and they have two basic options:

• Mandate particular measurement processes. For example, many firms stipulate that some people decisions, such as hiring above a certain level of seniority, are supported by the use of particular measures. Others insist on involving people who have attended specific training (such as advanced interviewing skills).
• Guide use. Some organizations provide lists of recommended tools and vendors to ensure overall consistency while still allowing business units some flexibility in their choices.

The route businesses choose will depend on their size and culture. What makes ensuring proper use so challenging for some businesses is the fact that it requires consistency. For larger, decentralized, or geographically dispersed organizations, this can be tough. Moreover, it touches on the activity and decisions of individual managers and leaders, which can be quite politically sensitive in some firms. And, of course, the key to being able to ensure proper use is having access to expertise, which many companies do not.

Yet despite the difficulties, businesses need to act to ensure that measurement outputs are used properly and to best effect. We return to this topic in the next chapter. For the moment, though, what is important is to acknowledge that it is a crucial issue and one that must not be ignored.

Strategy

As we have noted, the development of measurement practices in most businesses is not a strategic affair. It is emergent and demand led, and by and large, this is fine. In fact, it is essential that measurement processes serve a real business need.

The closest most businesses get to having a measurement strategy is when they consider one of three main issues: whom to assess, centralization, and standardization. Yet as we will go on to show, there is a fourth issue as well; it is rarely raised but has the potential to be the most strategic one of all.

Policy

As words go, policy hardly instills excitement. It certainly lacks the appeal of its sexier sibling, strategy, but it is no less important for that. Done well, creating policy is about giving simple, succinct, and clear guidance about what to do in certain situations.

We should state now that we believe policy should be kept to a minimum, and we are not fans of long policy documents. Thick manuals that lie unread are of little use. Brief, specific statements of principle, however, can be useful. At the very least, you need to be aware of some of these issues and know your position on them.

We tend to distinguish between two types of policy: those that emerge, developing out of issues as you first encounter them, and those that it is best to think through before any issues emerge. Emerging policies can be things like whether to retest people who report problems completing online ability tests or how to respond to some disability matters, such as using intelligence tests with people who are dyslexic. With these issues, the vendor is usually best placed to advise on how to respond. Companies do not need to have a stance on them beforehand, but they do need to learn from them when they do occur.

The policy issues that should be thought through before using measurement mostly concern data privacy and protection—in other words, who has access to individuals' measurement results and how these results are kept secure. The laws of the countries in which firms operate determine what they have to do here. One thing worth noting, though, is that there are some big national differences.

For instance, the United States may be far stricter about adverse impact than other countries (see chapter 5), but the European Union (EU) is far stricter about data privacy. Specifically, the 1995 Directive on Data Protection by the European Commission states that personal data may be transferred to other countries only if the countries provide adequate data protection. And according to the EU, the United States does not, so a US business with a European office or using an EU-based vendor could be restricted from sending data from the EU to the United States. Fortunately, there are some easy solutions that vendors can use, but you need to be aware of the regulations and comply with them.

Even companies that operate only in one location need to think about data privacy. For example, they need to consider how they will store data and keep them secure (larger firms may have a digital security department with its own policies on this). They also need to think about who will have access to measurement results. These can be simple decisions, but decisions do need to be made.

Larger companies tend to have a legal department to help them navigate these matters. For smaller firms, the easiest option is often to ask vendors for guidance. Since regulations differ, we cannot offer any definitive advice here, but as general guidance, we always try to follow four simple rules:

• Keep data secure—for example, lock reports in a cabinet, or password-protect a spreadsheet containing results.
• Give access to as few people as possible.
• Keep only the data you need.
• Keep data only as long as you need them. Where possible, make them anonymous.

One final factor you should think through before embarking on measurement processes is whether to provide feedback about measurement results to participants. When the measurement is part of a development process, the answer will almost always be yes. But with recruitment processes, it is not always so easy. Some companies see it as an ethical obligation. Others avoid it for fear of the risk of litigation. Some provide information only to successful candidates. Some do different things at different levels. And the feedback given can involve anything from a simple pass-or-fail statement to a detailed report and coaching session.

Factors influencing your decision will be the litigation environment of where you are operating and issues such as the number and level of candidates. There is no correct answer here, but our general advice is to give some level of feedback wherever possible. If nothing else, research has shown that when done sensitively, it can improve candidates' perceptions of the selection process and thus the reputation of the company.⁴ Furthermore, whatever level of feedback is given, candidates should always be advised beforehand what to expect.

Problems in Measuring Performance

In some roles, performance can be measured through results, for example, sales figures. Yet in most roles, there are no such straightforward results that can be used. Even when there are, using hard results as an indicator of performance is not always as perfect as it sounds. In sales jobs, for instance, there is the issue of whether it is fair to compare people who operate in different areas. As a result, researchers and vendors have tended not to use real results, but rather managers' performance ratings of people. Again, this may sound like a good solution, but it is not, for three reasons: lack of performance data, inaccurate performance ratings, and the fact that performance ratings can reflect many things.

Measuring Multiple Dimensions of Performance

For vendors to be able to develop better tests and more effectively target measures, businesses have to start measuring more than just overall performance. Indeed, if they want to be better able to predict success and thereby improve their people decisions, this is arguably just as important as the need to start measuring fit. The big issue, of course, is how to do it.

Managers have increasingly broad spans of control, which means they have to invest a significant amount of time in performance management. The tendency has thus been to try to simplify and accelerate the process for them. Indeed, we recently saw an article in which a company boasted of having an appraisal form that takes a mere seven seconds to complete.¹⁰ Putting aside the issue of whether this is something to aspire to, the reality is that any suggestion of greater complexity in the performance management process is likely to evoke concern. We want to be clear here. What we are proposing need not, and should not, be either complex or time-consuming. It need not add much extra work, but it does have the potential to make the ratings far more meaningful.

In fact, there is already a move to do this, especially in the United States. Companies are increasingly looking to monitor and reward employee behaviors that are discretionary and beyond their formal job descriptions that add value to the organization. They are called different things, such as organizational citizenship behavior, prosocial organizational behavior, extra-role behavior, and contextual performance.¹¹ But they mostly refer to the same thing—typically behaviors that contribute to making the workplace a better or easier place to work. Researchers have developed some detailed definitions and complex measures to help assess these behaviors. But managerial ratings need not be detailed or complex. They just need to be good enough.

Another increasingly common practice is competency-based performance management. This involves distinguishing between the results people achieve and the way they go about achieving them. This how element is usually measured through competencies, and it often contributes to 10 or even 20 percent of the overall performance score. This is definitely a step in the right direction. But the way it is implemented means that the data are often not useful for developing better talent measures. Managers are typically asked to rate a large number of competencies (we know one firm in which they have to rate eleven), which means the accuracy of the individual ratings tends to be low. Or they simply give an overall score for “behaviors,” which is too broad to be useful for our purposes here.

Measuring What You Value

The final foundation we are suggesting, then, is replacing the overall performance rating with three or four more specific ratings. The types of performance you measure should be driven by what you value as a business. Other types of performance sometimes measured in addition to task performance include safety-related behaviors, contribution to team performance, and self-development (the use people make of training). Importantly, these aspects of performance should not just be behaviors from a competency model. This is because competency models tend to change every few years, and ideally you want to be able to build up a database of results over many years.

If you are unsure where to start, you can simply measure the following three dimensions (see figure 6.1):

• Task performance (how well objectives are fulfilled)
• Organizational citizenship behavior
• Contribution to team performance

Figure 6.1 Dimensions of Performance

An overall performance score can then be produced by combining the more specific scores, perhaps using a weighting scheme. For example, if you believe that task performance is the most important aspect of overall performance, then you could create an algorithm that reflects this. The rating for task performance could account for, say, 70 percent of overall performance, with contribution to team performance making up 20 percent and citizenship behavior contributing 10 percent.

Of course, as we suggested earlier, these kinds of data are of limited use unless you collect them centrally and connect them with other data. You need to link performance information to results from talent measures. Doing this will enable you to better understand what you are predicting with the talent measures that you use and more precisely ensure that you are measuring all of the factors you need to. But most important, by being more specific about what type of performance you are predicting, you will be able to achieve higher validity levels and thus be better able to accurately identify and measure talent.

Not all organizations will be in a position to implement such a change in how they measure performance. But it need not be complex or time-consuming, and it holds the promise of better measures, better tools, and better people decisions.