Behavioral item—A word, phrase, or sentence that describes some aspect of behavior.
Cluster analysis—A statistical analysis that groups like objects or items.
Concurrent validity—The relationship between scores on an instrument and performance or effectiveness measured at about the same time the instrument was completed.
Construct—An abstract variable that cannot be directly measured.
Construct validity—The ability of the instrument to measure phenomena that are hypothesized to exist but for which we can have no direct measure.
Content validity—The degree to which the items are a representative and a comprehensive measure of the phenomenon in question.
Correlation coefficient—A number, resulting from statistical analysis, that indicates the strength of a relationship between two measures.
Criterion variable—A behavior or phenomenon separate from, but presumably related to, ratings produced by the instrument.
Cronbach’s alpha—A measure of the internal consistency of scales, based on the average correlation among items and the number of items; if this coefficient is low, either the scale contains too few items or the items have little in common.
Differential item functioning (DIF)—A statistical analysis that tests equivalence of an instrument for two populations that have equal abilities.
Empirical—Capable of being demonstrated or verified by scientific (usually statistical) methods.
Face validity—Whether the instrument looks like it measures what it is intended to measure.
Factor analysis—A set of statistical approaches to grouping items or variables into categories known as factors.
Feedback—Information about a person’s performance or behavior, or the impact of performance or behavior, that is intentionally delivered to that person in order to facilitate change or improvement.
Feedback display—The presentation of scored data, represented graphically or in narrative form.
Graphic display—Charts or graphs that provide a visual numeric portrayal of data.
Halo effect—High correlations resulting from raters’ overall bias toward the individual.
Hypothesis—An expectation or prediction about observable phenomena derived from theory.
Independent criteria—A criterion variable that comes from a different source or is external to the instrument itself.
Internal consistency—The average correlation among items included on the same scale.
Internally homogeneous—Having high internal consistency.
Interrater reliability—Agreement among ratings within rater groups.
Item—A word, phrase, or sentence that describes an aspect of behavior, performance, skill, or personal characteristic.
Item content—The behavior, skill, or characteristic to which an item refers.
Item level—Referring to each item individually.
Item response theory (IRT)—A statistical analysis that tests whether members of different populations (for example, English and French), who are assumed to be “matched” on the abilities measured by an instrument, answer the questions in a similar fashion. Examinees are matched using the item score patterns.
Item-scale correlations—The relationship(s) between responses to individual items and computed scores on scales.
Logistic regression (LR)—A statistical analysis that tests the equivalence of an instrument for two populations by using the total instrument score (or a score adjusted by deleting questionable items) to match examinees.
Mean—Arithmetic average of a set of data points or numbers.
Model—A set of hypotheses about the relationships between variables or constructs.
Multimethod study—A study that compares the correlation between two constructs measured by different methods (for example, rater observation and paper-and-pencil methods). The correlation between the two different constructs measured similarly should be lower than that between a single construct measured using different methods.
Narrative display—Data presented in text format, usually with personalized interpretation.
Norm group—Group of managers whose scores are stored in the vendor’s test database and are output as a comparison on every individual feedback report.
Norms—The average scores of all people who have taken the instrument or the average scores of specific target groups.
Population—The whole universe or group from which samples are drawn.
Predictive validity—The relationship between scores on an instrument and performance or effectiveness measured at a future time.
Psychometrics—The study or practice of statistically measuring aspects of performance, behavior, personal characteristics, and other human attributes.
Rater groups—Groups of individuals providing responses to items on an instrument.
Raters—Individuals providing responses on an instrument.
Rational/intuitive scale development—Grouping items together based on the authors’ expectations or experience about how different skills or behaviors relate to one another.
Response scale—A set of response choices provided on an instrument for raters to use in evaluating behavior, performance, or skill.
Sample—A subset of individuals drawn from the total population or database.
Scale—A group of items that have logical and empirical coherence.
Standard deviation—A statistical index indicating the amount of spread in the distribution of scores around the average score of a sample.
Statistical scale development—Using statistical-grouping procedures such as factor analysis, cluster analysis, or item-scale correlations to sort items into scales based on the degree of similarity in the response patterns of raters.
Target population—A group, sometimes with particular characteristics, for whom the instrument is intended; the group that will be rated on an instrument.
Test-retest reliability—Stability of scores over short periods of time.
Validation strategy—The process or set of studies used to collect evidence regarding the uses to which scores from an instrument can be reasonably and meaningfully applied.
Validity—The extent to which an instrument measures what it is supposed to measure; the appropriateness of the inferences made about scores from an instrument.
Variable—Any measurable entity whose value is not constant or given.
Weighting—Assigning differential value to points on a scale.