13.4. Summary

This chapter summarizes the importance of and methods for establishing the reliability and validity of rating scales. Quality quantitative scientific research must include the use of measurement tools that have been validated for the population under study. The validation of an instrument is not a single test, but a summary of multiple psychometric properties. That is, a scale should have acceptable levels of internal consistency,test-retest reliability, inter-rater reliability, convergent, divergent and discriminantvalidity, as well as responsiveness.

A rating scale is said to be reliable if multiple measurements on a subject agree. Aspects of reliability presented in this chapter include internal consistency, test-retest reliability, and inter-rater reliability. A measure is said to be valid if it accurately reflects the concept it is intended to measure. Validity discussions included in this chapter include convergent, divergent, and discriminant validity, as well as responsiveness and assessment of the factor structure of the instrument.

Throughout the chapter, multiple statistical methods for assessing the various aspects of reliability and validity are presented. The statistical methods are presented along with examples from clinical trials, diagnostic studies, rater training sessions, and scale validation studies. Multiple examples include SAS code and output to facilitate understanding and ease of application of these concepts.