Appendix D

Research Methodology

Purpose

The purpose of this research project was to develop a valid set of roles, competencies, and sales areas of expertise (AOEs) relevant for sales team members. The roles, competencies, and AOEs were intended to be comprehensive enough to address both emerging and current responsibilities. To that end, the final model was developed based on a multi-perspective and multi-method approach to avoid bias from any single information source.

Although the model is intended to be comprehensive, it must be specific enough to identify the particular skills, knowledge, and outputs necessary to achieve world-class selling excellence. The model’s ultimate effectiveness is defined by how it will be used (for example, workforce planning, guiding professional development, providing personnel selection criteria, determining eligibility for leadership development programs, and assessing the impact of training).

All of the competencies in the model are important for success, but an individual might perform only one role or multiple roles, depending on his or her position, tenure in the profession, and size and maturity of the sales organization. If practitioners effectively demonstrate all the competencies in the model, they should be considered highly valuable for organizations that hope to promote sales professionalism in the workplace.

Approach

To create the ASTD World-Class Sales Competency Model, the project team followed a four-phase process: data collection, model development, validation, and refinement and confirmation. See Figure D-1 for a graphic representation of the basic development process.

Phase 1: Needs Assessment and Data Collection

The objective of this phase was to finalize project requirements and define a solid research plan that replicated the high standards and past successes of ASTD competency modeling efforts (see Appendix C for a history of these efforts). The project also leveraged a solid partnership with Productivity Dynamics, which possesses a proven methodology that follows best practices in competency modeling for numerous sales-specific roles.

The first phase consisted of three parts: expert interviews, literature review, and a review of existing competency models. One of the goals of the research was to build an international model. Every effort was made to ensure a global perspective: The literature review included international sources, the expert interviews were conducted with a worldwide sample, focus groups were conducted in multiple countries, and advisory panel members were drawn from around the world.

Expert Interviews

Analysts gathered information from job incumbents, executives, and thought leaders regarding current and future job competencies and role requirements. This approach focused on the strategic direction of sales jobs within the context of trends affecting the profession.

Most input was gathered from the World-Class Sales Competency Advisory Panel, a group of leading sales experts whose role was to provide wisdom and guidance to the project research team. Advisory Panel members represented

• companies engaged in business-to-business selling
• direct, indirect, and technical selling activities
• multiple industries engaged in business-to-business selling (financial services, information technology, telecommunications, pharmaceutical, public sector and education, and management consulting)
• sales team members, sales operations members, sales managers, and sales trainers or consultants.

Advisory panel members were asked the following questions:

• What are the major tasks and categories of work performed by sales profession role representatives?
• What are the outputs produced?
• What are the quality indicators for these outputs?
• What competencies are needed to complete the tasks or work?
• What are the necessary skills and knowledge domains that support the competencies?
• What trends and forces are currently shaping the sales profession?
• What forces or issues are pertinent and “front of mind” for professionals in each role category (that is, performance, ethics, globalization, and talent management)?

Literature Review

Analysts conducted a search of current international publications, studies, and websites to ensure an adequate analysis of competencies, roles, and areas of expertise. They conducted an environmental scan and literature review to determine major trends and forces shaping the sales profession as well as current best practices in sales competency modeling and worked to identify pivotal roles in the sales profession.

Review of Existing Competency Models

Analysts obtained and reviewed existing sales competency models, skills inventories, and related documents from various organizations, associations, and public sources.

At the conclusion of Phase 1, a summary report was drafted, presenting the findings. This report became the major input for phase 2, model development.

Phase 2: Model Development

The objective of this phase was to create a draft of the competency model for the sales profession that covered pivotal sales roles. This information was garnered through a series of in-depth interviews with subject matter experts—people with exemplary performance as sales professionals who represented

• top performers and managers in multiple roles within each of the role categories identified
• business-to-business sales efforts
• direct selling, indirect selling, technical selling, and sales management functions
• multiple industries (financial services, information technology, telecommunications, pharmaceutical, public sector and education, and management consulting).

Phase 2 comprised the following steps: data integration, expert group interviews and content review, creation of a Draft 2 Model, a special session at the 2008 ASTD International Conference and Exposition, final expert review, and the creation of a Draft Model 3.

Data Integration

The data integration step was conducted over a three-month period. A team of analysts focused on identifying best practices in sales competency modeling as well as existing models and relevant literature on the topic. Activities in this step included

• reviewing existing competency models and related documents to extract information related to sales exemplary performance, sales competencies, trends, and forces that shape the sales profession
• conducting interviews and facilitating meetings with advisory panel members
• defining an architecture for the sales competency model
• populating the model with existing competencies.

Simultaneously with integrating the feedback of subject matter experts on pivotal sales roles, analysts worked to define the sales competency model architecture. Their goal was to develop a straw model that was shaped by the steps in phase 1, as well as input from advisory panel members. The roles and competencies were defined as fully and as clearly as possible using sample outputs. The competencies included a brief definition and key actions.

Individual competencies in the model were specifically mapped to at least five other major competency studies. This approach ensured that the competencies identified were derived from consistent and powerful trends in findings across multiple publications and studies. In some cases, analysts reviewed existing competency models from other organizations. From these inputs, the first draft model was created.

Expert Group Interviews and Content Review

Two consultants conducted one-on-one interviews with 115 participants, each lasting between 30 and 50 minutes, covering the following set of core questions (with opportunities for follow-up questioning):

• Does the list of competencies address all of the major competencies required for people working in the sales profession? If not, what competencies should be added? Why?
• Should any of the competencies be removed, combined, or subdivided?
• Are they sufficiently discrete? What should be changed? Why?
• What specific edits do you have for competency titles, definitions, or key actions?
• Does the list of roles cover all of the major roles for the sales profession? If not, what roles should be added?
• Should any of the roles be removed, combined, or subdivided? Are they sufficiently discrete? What should be changed? Why?
• What specific edits do you have for role titles, definitions, or representative actions?
• Does the list of areas of expertise address most of the major specialty areas in the sales profession? If not, what AOE should be added? Why?
• Should any of the AOEs be removed, combined, or subdivided? Are they sufficiently discrete? What should be changed? Why?
• Please address one or two of the AOEs in which you have expertise. For each AOE you are reviewing, what specific edits do you have for titles, definitions, key knowledge areas, or key actions?
• Are the key knowledge areas comprehensive? Would you suggest any additions, deletions, or edits?
• Are the key actions comprehensive? Would you suggest any additions, deletions, or edits?
• Please list some of the common outputs a person would produce when using the knowledge and skills associated with each competency.
• What are the quality indicators for these outputs?
• What trends and forces are currently shaping the sales profession?
• What forces or issues are front-of-mind for professionals in each role category?

Draft Model 2

The objective of this step was to define a complete model. Activities included

• using qualitative data analysis techniques to analyze the results of interviews and defining a set of core competencies for the sales profession
• populating the architecture and straw model developed in the first step of phase 2 and preparing a comprehensive next iteration (Draft 2) of the model
• sharing Draft 2 of the model with stakeholders and members of the advisory panel to obtain feedback
• revising the model iteratively as feedback was received from reviewers
• finalizing Draft 2 of the model.

ASTD 2008 International Conference and Exposition Special Session

In a special session at the ASTD 2008 International Conference and Exposition in May 2008, attendees were shown the draft model and then broken into subject matter teams to discuss specific sales areas of expertise. They were asked to share their thoughts on each AOE and to identify any missing information or unclear terminology. Each AOE had an advisory panel member available to discuss methodology and content. Comments were compiled and summarized by analysts within two weeks.

Final Expert Review and Draft Model 3

The goals of this step were to validate Draft Model 2 and the data collected in the first and second phases of the project and to determine the relative importance of the competencies across the various roles. The third draft of the model was created by validating the model through 20 virtual and face-to-face focus groups in multiple countries and locations.

Analysts conducted interviews with a total of about 200 people who reviewed the existing model content. Participants were given 30 minutes to review the model for accuracy and provide suggestions for refinement in their areas of content expertise. A partial list of these experts and others who contributed to the study can be found in Appendix F.

Participants were selected to provide broad expertise in selling, sales operations, sales training and development, and sales management. They were asked open-ended questions designed to determine whether the core competencies, roles, and areas of expertise described in the model were comprehensive, accurate, and understandable. They were also asked to review the key knowledge areas and key actions associated with each competency and role, to list the common outputs and quality indicators for sales team members, and to identify the major trends and issues facing the sales profession. These inputs were used to create Draft Model 3.

Phase 3: Model Validation

The objective of this step was to use a quantitative research approach to further validate the model and determine the relative importance of the sales profession AOEs and competencies for various roles and job titles. Activities in this step included

• developing a survey to align the emerging competency model with the pivotal roles
• identifying appropriate demographics and related variables (role, industry, and tenure) to stratify, then select the sample
• implementing the survey leveraging ASTD resources
• utilizing quantitative data analysis techniques to analyze the results and capture the relative importance of the competencies and AOEs for various roles and job titles.

A survey instrument enabled the team to collect background information from participants and ask them to indicate relevance and desired proficiency level of behavioral indicators for each competency using five-point Likert scales. The results from this survey can be found in Appendix E.

Analysts administered the survey on a website using third-party software. The survey mirrored the structure of the ASTD competency model and its content. Survey items were designed to address a list of critical questions posed by the research team, such as the relative importance of the competencies and time spent performing each AOE.

All survey participants were asked to rate roles and competencies in terms of importance and frequency of use. Participants were also asked to indicate their primary area of expertise and then were prompted to rate content appropriate for their expertise. The competencies, roles, and AOEs presented in the survey were based on the refined model developed from the information gathered in phases 1 and 2. Copies of the survey are on file at ASTD.

The final sample size included 2,128 responses, representing 5.9 percent of the targeted population.

Demographics from this sample are similar to the demographics of ASTD’s actual membership. The margin of error for conclusions is +/- 5 percent. This means that if the survey were to be conducted again, there is an extremely high probability that the results would be exactly the same. The margin of error for this study is in line with commonly accepted standards for survey research. Demographics of the survey respondents can be found in Appendix E.

Phase 4: Final Refinement and Confirmation

Project Team Refinement and Advisory Panel Approval

Using information collected from the survey results, analysts from Productivity Dynamics created a final model for approval by the ASTD project team and advisory committee, which was the same as the final draft model presented to survey respondents. Analysts reviewed survey responses using specialized software to determine if any content should be dropped from the final model due to invalid responses or a lack of data. No content was dropped from the final model.

Creation of the Final Model

The final draft of the model was presented to the project team and advisory committee in December 2008. No revisions were suggested. After the model was finalized, the project team began to draft the structure and content of this report.

Validation and Final Report

The objectives of this step were to finalize the competency model and to prepare and issue the final report. Activities in this step included

• developing the validation survey based on role-specific and job-specific competency priorities and cross-role and cross-job similarities
• administering the validation survey and collecting the results
• preparing this final report, which includes the methodology and results of the validation survey.

Using information collected from the survey results, ASTD created a final model for approval by the ASTD project team and advisory committee. It included slight modifications made in response to a content analysis of written comments from survey respondents. Each written comment was evaluated in terms of its relationship to the model and how frequently it was mentioned. Changes were reviewed in a group setting and approved by subject matter experts when appropriate. Usually, any revisions represented clear omissions from the model. These revisions included clarifying words for particular key actions or key knowledge areas. All of these revisions are reflected in the Competency Dictionary in Appendix A. Copies of the final revisions are on file at ASTD.

A meeting was held with the ASTD project team and advisory committee to review the final competency model structure, gather feedback, and suggest revisions. Minor revisions were made to the model graphic. After the model was finalized, the project team began to draft the structure and content of this report. The end product represents the systematic development and phased process for creating a well-constructed and validated sales profession competency model.