Reliability and Validity of Interviews
Interview validity refers to whether interview questions map to specific competencies listed on the Competency Assessment Worksheet. To ensure maximum validity, there needs to be a one-to-one correspondence between interview questions asked and underlying competency.
To ensure interviewer reliability, all interviewers need to use interview questions that are designed this way. It is only when this mapping has been documented and used by all interviewers for a particular candidate can it be said that the interview data for that candidate is reliable and valid, or consistent with the competencies deemed essential for the job.
According to David Cherrington in The Management of Human Resources Instructor's Manual, the “unreliability of employment interviews is created by several factors that reduce the consistency of the information obtained from them…. The mood of the interviewer may be different at different interviews, the questions asked in the interview may differ, the mood of the interviewee may be different, the topics discussed will change each time—in short, because each interpersonal interaction is unique and can never be exactly duplicated.” Cherrington suggests the following steps for increasing the reliability of interviews:
Ask systematic and well-structured questions and avoid vague, open-ended questions that invite the candidate to provide hypothetical answers.
Make sure that permissible responses can be clearly defined and properly categorized during the candidate assessment process that will occur later on.
To the extent possible, consider developing a rating scale that can be used to sort and rank responses against “best in class” (this is made considerably easier if behavioral interviewing techniques have been used).
Reliability is a necessary but not sufficient condition for developing high-quality interview questions. Establishing reliability ensures consistency; establishing interview validity ensures that the questions asked map to the appropriate competencies.
There are four types of validity that can be used to rate the quality of proposed interview questions. They are described below.
Face validity. An interview question has high face validity if it appears to a candidate to measure what it is supposed to measure. For example, Joshua Devins' request that the candidate for the HRD Director's job provide a “brief summary of his or her relevant work experience” doesn't have high face validity because it is too vague. An improved question would have focused on one of the specific skills required by the HRD Director.
Content validity. An interview question possesses content validity when a group of recognized content experts or subject matter experts has verified that the question assesses an appropriate competency for the particular job or position. Content validity must be established before the interview begins; it is too late to attempt to establish it during the interview itself.
Concurrent validity. Concurrent validity refers to the ability of an interview question to correctly classify qualified and unqualified job candidates. Establishing concurrent validity is the “holy grail” of interviewing, and as such, much is at stake when attempting to design questions that have concurrent validity. Perhaps the best approach is to rely on the research involved in separating exemplary from non-exemplary performers prior to doing behavioral interviewing.
Predictive validity. Predictive validity in interviewing refers to the ability of specific questions to predict future success on the job. The only sure way to establish predictive validity is to interview the hired candidate and ask him or her which interview questions provided the most accurate gauge of what the job actually involves.
How to Chose the Type of Validity
While it is critical to establish the validity of all interview questions, it is not practical to design interview questions to reach all four types of validity. Generally speaking, the following guidelines should work in most situations:
Because it is relatively easy to establish, cost effective, and offers a solid defense in case of adverse legal action, establishing each question's face validity should be a requirement for all interviewing.
Establishing concurrent or predictive validity, while certainly desirable, can be time-consuming. Consequently, most organizations rely on content validity, believing that questions with high content validity offer the highest value for the time and cost involved in developing them.
Do Your Homework
Ultimately, the key to a successful interview process is for the manager to do his or her homework. This entails having a strong understanding of the corporate culture, completing the Competency Assessment Worksheet, and determining the most appropriate interview method for your needs. Lastly, the manager needs to find a keen balance between job fit and culture fit. Once those tasks are successfully completed, the manager is sure to find the best candidate for the job.