STEP 2:
COLLECT A COMPLETE SET OF MATERIALS

When you have identified several instruments you wish to evaluate, you need to obtain five pieces of information about each of them. You cannot make an informed decision using only a copy of the instrument or a promotional brochure.

Specifically, for each instrument you wish to consider, you should obtain the following:

•  A copy of the instrument itself. If the instrument has one form for the individual to rate himself or herself and a separate form for the others who will rate him or her, get both.

•  A sample feedback report (a representation of what the manager will receive after the instrument is scored). You can’t tell what type of feedback your managers will actually receive by looking at the instrument they will fill out. The sample could be a complete report, or it could be part of a report such as an example of the feedback display in the technical or trainer’s manual. Either type will do.

•  A technical manual or other publication that outlines in detail the developmental and psychometric research done on the instrument.

•  Information about any supporting materials that accompany the scored feedback, such as interpretive materials, development guides, goal-planning materials, and the like.

•  Information about price, scoring, and whatever certification or training may be required to purchase or use the instrument.

It is not at all unreasonable to request this quantity of information. American Psychological Association guidelines (APA, 1985) require that this information be available upon request when an instrument is offered for sale.

In addition to seeking the recommended information, you should, through all the steps that follow, look for evidence of a commitment to continuous improvement on the part of each instrument’s developer. This is especially true if an instrument has been around for awhile. As we will discuss in the section on validity, research should always be in progress, because no instrument can ever be considered valid once and for all. Expect revisions in the scales over time; these are often made when additional validation studies have been completed. Expect revisions in the presentation of feedback as well; these are often made as the developer learns from the experience of those who have used an instrument. It is not uncommon for good instruments to have more than one copyright date, because even small revisions to content can cause changes in other areas, such as scale weightings or instrument norms.