Item Generation
Initially, 155 items, based on a content analysis of the data collected in the original interviews, were written to tap the core elements of developmental jobs. Verbatim responses were used as guides for writing the items; thus many items reflect the terminology and concepts used by the managers who responded. The items took the form of descriptive statements about the job. Respondents were asked to rate each of them on a five-point scale ranging from “not at all descriptive” to “extremely descriptive” of the job.
Creation of Job Component Dimensions
We had no preexisting idea for how the items should be grouped; thus we used an empirical approach. Managers from various organizational levels completed the JCP about their current jobs, and the following data analysis steps were used:
1. Item response distributions and item standard deviations were examined to ensure that responses to each item varied adequately across raters.
2. Items were subject to a series of factor analyses to see how items clustered into job component dimensions.
3. The clustering was further refined by examining item-dimension correlations, iterative Cronbach alpha coefficients, and conceptual fit with other items.
4. The resulting dimensions were tested again with a new sample of managers using confirmatory factor analysis.
At this stage, 113 items were retained to use in the Job Challenge Profile. These items clustered into fifteen job component dimensions. (See McCauley, Ruderman, Ohlott, & Morrow, 1994, for a further description of this version of the JCP.)
Psychometric Evaluation
Further data were collected to evaluate the reliability and validity of the job component dimensions. We assessed both internal consistency and test-retest reliability. Validity evidence was built using several types of analysis:
• We tested whether managers who rated their jobs higher on a job component also rated their jobs as more challenging and more developmental.
• We examined whether managers who rated their jobs higher on a job component reported higher levels of learning specific lessons from their jobs.
• We assessed the degree to which different types of jobs (line vs. staff jobs, jobs at different organizational levels, domestic vs. foreign assignments) could be distinguished based on the job component ratings.
• We tested hypotheses about how the various job components would be related to various outcome measures (ratings of task quality, experiences of stress on the job).
The results provided strong evidence for the reliability and validity of the job component measures (see McCauley et al., 1994).
Refinement as a Management Development Tool
Although the Job Challenge Profile was established as a measurement tool for continued research about on-the-job learning, it needed to undergo further refinement to maximize its usefulness as a tool in management development. The questionnaire needed to be shorter, more focused, and easily scored. The authors, based on our experience with the questionnaire, identified job component dimensions that were problematic or less central to the JCP and eliminated these. Next, a series of analyses conducted with JCP data collected from one thousand managers helped identify the best items to retain. These analyses also provided a possible restructuring of several of the dimensions. Finally, additional qualitative research conducted since the original items were written suggested two new job components (both focused on managing diversity) to add to the questionnaire.
This refinement process led to the current fifty-item, ten-dimension, self-scoring version of the JCP. The fifty items, grouped by dimension, are shown in Exhibit 1.
The ten dimensions (most of which were very similar to dimensions of the earlier 113-item version) were reevaluated for evidence of reliability and validity. The results of these analyses, presented in greater detail in Appendix B, provided positive support for the new version of the JCP.
Exhibit 1 Items on the Job Challenge Profile
Unfamiliar Responsibilities
• You lack experience important to carrying out some aspect of your job (for example, financial or market analysis, negotiation, or budgeting).
• You have to manage something, such as a function, product, technology, or market, with which you are unfamiliar.
• Others question whether you are “ready” for this job.
• Compared to previous job incumbents, you do not have the credentials, background, or experience expected for this job.
• This job is no less than a change in your career direction, that is, you are doing a type of work dramatically different from what you have done before.
New Directions
• You have to carry out a major reorganization as a result of a merger, acquisition, downsizing, or rapid growth.
• You have to make major strategic changes in the business—its direction, structure, technology systems, or operations.
• You are trying something the organization has never tried before; no one knows for sure how to do it or how it will come out.
• This job includes launching new organizational ventures, such as new product lines or acquisitions, new functions or groups, new plans or concepts, or new facilities.
• You have to create or establish new policies or procedures.
Inherited Problems
• You inherited widespread morale problems.
• You need to restore the credibility of your unit with the rest of the organization.
• To succeed in this job you have to dismantle the strategy your predecessor had established.
• Your business or unit has a record of poor performance.
• You must solve major problems a predecessor created.
• Your direct reports resist your initiatives.
• There is an interpersonal conflict between you and at least one of your key direct reports.
• Your employees are used to doing things the way they have always been done and are reluctant to change.
• Key members of your staff are incompetent, demotivated, technically obsolete, or otherwise performing poorly.
• Some of your key direct reports lack the experience to do their jobs without close supervision from you.
High Stakes
• Your success or failure in this job will be evident to higher management.
• You are responsible for decisive action in a highly charged environment.
• You are being tested by higher management.
• There are clear deadlines by which your key objectives must be accomplished.
• There is pressure to complete a major piece of your job quickly.
Scope and Scale
• This job is a dramatic increase in scope for you (managing significantly more people, dollars, sites, functions, and so forth).
• The job is potentially more than even a good delegator can handle.
• You are responsible for numerous different products, technologies, or services.
• You are responsible for multiple functions or groups.
• This job puts you under constant pressure; there are seldom any periods to “catch your breath.”
External Pressure
• The customer base you work with is extremely varied.
• To achieve your most important goals, you must influence people outside the organization (for example, clients, suppliers, unions, or government agencies).
• You manage relationships with government officials or regulatory agencies.
• You must deal with diverse clients, customers, or markets.
• You have to carry out formal negotiations with an outside body, such as a union, client, or joint venture partner.
Influence Without Authority
• You have to coordinate action across dispersed sites over which you have no direct authority.
• To achieve your most important goals, you must influence peers at similar levels in other units, functions, divisions, and so forth.
• Achieving your goals depends on how well you handle internal politics.
• To accomplish a major portion of your objectives, you must influence and work with executives higher than your immediate boss.
• A great deal of coordination with other organizational units or functions is required.
Work Across Cultures
• You conduct business with people from different countries.
• Your job requires working in a foreign country in which the culture is different from your own.
• This job requires dealing with foreign companies, agencies, or governments that can have a substantial impact on the business.
• You manage parts of the business that are scattered across the world.
• Your job requires understanding the traditions and values of people from different cultures.
Work Group Diversity
• In terms of demographic variables, you have a diverse group of direct reports.
• You are part of a diverse work group.
• You are responsible for developing managers from both genders and different ethnic groups.
• You have to get people from different racial, religious, cultural, or ethnic backgrounds to work together.
• You must make personnel decisions about employees who differ from you in terms of race or gender.