APPENDIX B

Psychometric Properties of the Job Challenge Profile

A number of analyses were conducted to build evidence that scores on the current version of the Job Challenge Profile are reliable and are valid measures of developmental job components. Reliability and validity evidence for earlier versions of the instrument can be found in McCauley, Ruderman, Ohlott, and Morrow (1994).

Three samples of managers who completed the JCP were used in the analyses:

•  Sample 1 consisted of 1,143 managers who completed the JCP between 1992 and 1996 as participants in a management development program. These managers were from a variety of companies and were normally distributed by organizational level. There were almost equal numbers of men (51 percent) and women (49 percent) in the sample. This sample was the same sample used to calculate the relative scores (see Scoring the Questionnaire on p. 23).

•  Sample 2 consisted of 264 managers who completed an earlier version of the JCP as part of a research project. Again, these managers were from a variety of companies and were normally distributed by organizational level. There were more men (68 percent) than women (32 percent) in this sample.

•  Sample 3 consisted of fifty managers who completed the JCP at two points in time (four to six weeks apart). The majority of these managers were from one professional organization.

Descriptive Statistics

The tables below provide means, standard deviations, and intercorrelations for the ten job component dimensions.

Reliability

Internal consistency of the items in each job component was assessed by calculating alpha coefficients for Sample 1. The coefficients ranged from .63 to .80. The test-retest reliability coefficients calculated for Sample 3 indicated quite high short-term stability of scores on each component. Because jobs change over time, we did not examine nor expect longer term test-retest reliability.

1. Test-retest estimates are not available for these two components.

Validity

We expected scores on each developmental job component to be related to perceptions of job challenge and to how much job incumbents report learning from the job. To test these expectations, in addition to completing the JCP, managers in Sample 1 responded to the following two questions:

A.  Which option below best captures how you feel about the challenge in your job?

1 =  I feel bored from the lack of challenge.

2 =  I feel somewhat under-challenged.

3 =  feel about the right amount of challenge.

4 =  I feel somewhat over-challenged.

5 =  I feel overwhelmed from too much challenge.

B.  Relative to other jobs and assignments you’ve had, how much is your current assignment contributing to your growth as a manager?

1 =  As far as I can tell, I’m not developing in this job.

2 =  This job has some developmental elements, but I’m not getting a whole lot out of it.

3 =  This is a very developmental job, but I’ve had several that developed me more.

4 =  One of the most developmental jobs I’ve had so far. It’s in the top two or three.

5 =  Clearly the single most developmental job I’ve ever had.

Managers in Sample 2 also responded to the second question above. In addition, they were given a list a forty-one potential areas of learning (for example, leading employees or thinking strategically) and asked to what degree they were learning from their jobs in each of the areas. We averaged the ratings across these forty-one items to derive an index of learning.

The table below summarizes the correlations between scores on the JCP components and the criteria measures of job challenge, development, and learning. Four components (unfamiliar responsibilities, new directions, high stakes, and scope and scale) had the strongest correlations across all four criteria. Three components (external pressure, work across cultures, and work group diversity) were not significantly related to perceptions of job challenge; however, each was significantly related to one or more measures of development and learning. All of the components were significantly correlated with the index of learning.

*p<.05.

We also asked managers in Sample 1 whether their current jobs represented any of the following “firsts”:

•  First exposure to this part of the business;

•  First time managing other managers;

•  First time in a line position; or

•  First time in an international assignment.

We expected to find that managers who experienced a “first” in their current jobs would rate those jobs higher on “unfamiliar responsibilities.” In addition, we believed that each of the “firsts” would have certain developmental components that stood out.

The correlations in the table below indicate that managers with “firsts” do indeed rate their jobs higher on “unfamiliar responsibilities.” Those who moved to a part of the business that is new to them also rated their jobs higher on “influence without authority.” Those in their first line jobs also rated their jobs higher on the two Managing at High Levels of Responsibility dimensions: “high stakes” and “scope and scale.” Those in their first international assignments rated their jobs higher on “work across cultures.” Managers’ first times managing other managers were associated with a number of the job components: “inherited problems,” “problems with employees,” “high stakes,” “scope and scale,” and “work group diversity.”

*p<.05.

Finally, because we assumed that developmental job components stretch individuals beyond their current capacity, we expected that managers who experienced one of the job components would also experience some degree of stress in their jobs. Managers in Sample 2 rated the degree to which they experienced three types of stressful emotions in their jobs: challenge (such as eagerness or anticipation), threat (such as anxiety or worry), and harm (such as anger or disappointment).²

The table below shows that each component is associated with at least one type of stressful emotion. Note that many of the components are related to threat. This suggests that it is important to reassure managers that it is natural to feel anxiety as they learn from these job components; in fact, it could be a sign that they are being stretched beyond their current capacities. Note also that only one component, “problems with employees,” is related to harmful emotions. Harmful emotions are the more negative stressful emotions and are often felt when a manager experiences little control over a situation. Thus, although managers may feel threatened by a developmental job component, they usually feel they have the capacity to deal with it when they face it in their jobs.

*p<.05.

²We adapted this measure from Folkman & Lazarus (1985), Journal of Personality and Social Psychology, pp. 150–170. Note that “challenge” is more narrowly defined in this context than in our discussion of challenge as a defining feature of key developmental assignments.