STEP THREE

Collecting Relevant Data

OVERVIEW

Planning your data collection

Choosing among types of data collection

Identifying data sources

Maximizing data validity and reliability

As a result of the work you completed in Steps 1 and 2, you are ready to begin collecting the data needed for strategic planning decision making. Data collection is often the most time-intensive part of strategic planning. If it's not done wisely, valuable time will be wasted on unnecessary activities or error-ridden data will be used to make important decisions that will affect the organization's short- and long-term health. Although you want the data you collect to be as complete as possible, don't collect it simply because you feel you must or because it seems like a good idea. The effectiveness of the final strategic plan depends on the quality—more than the quantity—of the data used in building strategy tactics.

Data always are needed as soon as possible. Typically, you're under pressure to complete the strategy so you can execute the plan. Most likely, you have limited resources and want to spend your time implementing changes rather than gathering data. In our discussion of this third step in the planning process, we'll focus on the tools used to collect relevant data for developing and supporting the strategic plan. Specifically, we'll address the following topics:

  planning for data collection

  choosing among types of data collection

  identifying sources of preexisting information

  maximizing the validity and reliability of the data.

The goals of data collection are to help the organization make critical decisions regarding its strategy and to identify the tactics it should deploy to best meet its long- and short-term goals and objectives. The data collected should help prioritize work and resources and help build the case that executing the identified tactics is important to the organization's health.

In collecting data for strategic planning, most time is spent in identifying needed information, soliciting original data, and designing and testing collection instruments. In this step we'll provide resources to reduce the time you spend on these activities, thereby enabling you to complete Step 3 as quickly as possible.

Case Examples: Collecting Data

Each organization has different needs, and each strategic plan requires the gathering of different information. The following case examples demonstrate how unique sets of circumstances drove different approaches to collecting the data needed to develop a strategic plan.

Learning from Surveys and Interviews

Maintaining a competitive advantage is a key driver for American Vista Software, which supplies software for the computer design industry. It's a highly competitive field. In the past two years, the company has acquired 10 other firms and merged cultures and product lines. There are plans for more acquisitions.

During the business scan completed in Step 2, it became clear that the organization's infrastructure and process documentation were not well communicated. Although employees often discussed among themselves the lack of documentation and processes, they never had the opportunity to articulate the problem in a way that made senior management aware of the company cost of not having sound documentation and processes. Now, using surveys and interviews, American Vista is working to determine what tactics to use to document the processes easily, keep the documentation current, and ensure that employees know how to access and use the documentation. Although using an electronic performance management system has been discussed to answer the short-term need, senior management wants a long-term strategy for process development and documentation that encompasses all facets of the organization, from operations to sales to administration.

POINTER

Getting a Picture of What's Available and What's Needed in the Field

Capital Hills Insurance has been in business more than 60 years. During the past year, the company reorganized and completed several complex initiatives, including updating its overall technology plan. All of the field offices have new computer equipment, and all sales associates now have a personal computer. This was part of an outcome identified in the former strategic plan.

With this new technology, the strategic planning group decided that the sales materials and process would be more leading-edge if field personnel had sales literature and product information available on automated systems at their workstations. Before this is finalized as an objective for modernizing the salesforce, however, the strategic planning group needs to complete these data-collection activities:

  a literature search to identify the best practices in the industry and the practices used by key competitors?

  interviews with the sales management, marketing, regulation, and materials development departments to identify the best processes, and other information to incorporate in the process itself

  focus groups with the salesforce to identify what will work best for them and their clients, and how the process should work in the field—including the initial and ongoing training efforts.

Planning for Data Collection

Making plans for data collection typically includes these nine activities:

1. Defining the criteria and outcomes of the data collection.

2. Identifying who will use the information and how it will be used.

3. Determining what information is needed.

4. Deciding if there will be multiple uses for the data and, if so, what they are.

5. Determining when the data will be needed.

6. Determining the resources available to gather the information.

7. Identifying the approaches to be used to gather the information.

8. Identifying methods to ensure the data are reliable and accurate.

9. Deciding who will collect the data and what they will need to support them (training, technology, administrative support, and so forth).

The data collected should be appropriate for use in developing a strategic plan and should be valid and credible. Before you devise a plan to collect your data, be sure you've identified all of the data needed to build the strategic plan. In Step 2 of the planning process, your group created a list of the types of information it believed to be significant. At this point, finalize that list and select the most effective and expeditious ways to obtain the information.

Review what you learned from the business scan, including the SWOT analysis and the business drivers, and then answer the following questions:

  What information do you need to build a plan that is future oriented, links to your organization's business, and supports your customers' needs?

  How will you know if you have the correct information?

  Do you need to validate the information? If so, with whom and how?

  What information will help you make a case for the outcomes, goals, and objectives you have identified?

  What information can you collect that would be contrary to the outcomes, goals, and objectives you have identified?

  What information will help you make informed decisions specific to the strategic plan and its ultimate mission and tactics?

  What information about your customers is important to consider in building a strategic plan? Are your customers' demographics and product needs consistent with your plans?

  What information about your business partners is important to consider? Are there time or economic constraints relative to your partners that you need to take into account?

  What data did your stakeholders and sponsors provide that give you insight into the organization's direction to ensure that the tactical plans you build are visionary and appropriate to the business needs?

  What business initiatives discussed by the sponsors and stakeholders can the organization develop products, programs, and services to support?

  Is there missing information regarding your customers, stakeholders, business partners, or sponsors that is critical to building the strategic plan?

  Does one set of information (customer data, for example) support or contradict another set (business partners data, for example), and how is this important to the strategic plan?

When you believe you've identified all of the data that needs to be collected, develop a plan that documents how the data collection process will proceed. Ultimately, to collect data you should know

  what data will be collected

  the type of data that will be collected (past/present/future trends)

  who will be accountable for collecting the data

  how the data will be collected (the methodology)

  what will be done with the data when they are collected (how they will be used)

  why the data are important (or the rationale for collecting the data)

  any potential problems that may affect the validity of the data (small sampling population, potentially biased collector, easily misrepresented data, and the like).

Worksheet 3.1 will help you keep the answers to these questions in mind as you formulate a plan for collecting your data. When collection is finished, place the worksheet in the strategic plan binder for future reference if needed.

As noted in the example in Worksheet 3.1, subjective data may be collected through focus groups. Before you create your collection plan, decide if you should institute controls to ensure the integrity of the data—controls such as comparing multiple samples or benchmarking like organizations to compare the focus group data.

It's also important at this point to ensure that the data you are collecting will meet the outcomes, goals, and objectives of the strategic plan. You may find it helpful here to review Worksheet 2.1, which you completed in the previous step. Update it with any new information that links to your outcomes, goals, and objectives. If data you've recently identified doesn't align with the outcomes, then decide if it can be skipped or if you have overlooked an outcome needed for the strategic plan. This may require some reworking of what you completed in Steps 1 and 2 and an accompanying validation with your group and senior management, but more often than not, you'll find that everything aligns.

WORKSHEET 3.1

Building a Data Collection Plan

Instructions: Use this worksheet to keep track of all the data you will collect before you develop your strategic plan. Understanding the data you seek will help you devise an effective and speedy collection strategy. An example is provided for illustration purposes.

With the data identified and aligned to the outcomes, you are ready to create your system for managing data collection. In tool 3.1 you'll find questions to answer for each piece of information you will collect. Looking at this type of detail for every bit of data you collect often reveals issues or problems. If identified early, these issues and problems usually can be resolved or supplemented with other data. By planning ahead and addressing possible problems at the outset, you reduce the time you spend on data collection because you have to contact the data sources only once to obtain everything you need, not repeatedly to collect data that were missed the first time through.

TOOL 3.1

Questions for Defining the Scope of the Data to Be Collected

Factor	Questions to Define the Scope
Type of data	What is to be collected? Sample answers: market information, productivity statistics
Age of data	Are the data from the past or the present, or are they future trend projections?
Data measurement units	What is the method of measurement for the data? Sample answers: percentage, count, totals
Purpose of data collection	Why are these specific data needed? What is the rationale for gathering them?
Data insight	When the data are presented, what do we expect them to tell us?
Type of measure	Is the measure an input, a process, or an output?
Strategic outcome	To what outcome are these data linked?
Use of data	How will the data be used, and what will be done with them?
Individual(s) accountable for data	Who is ultimately responsible for collecting these data?
Location for data collection	Where will the data be collected? Sample answers: on site, via the Internet, at a scheduled meeting
Method of data collection	What method will be used to collect the data? Sample answers: a system count, an existing or developed productivity report, a survey, a focus group, an industry study. How will the collection be completed? Who will develop the survey? What criteria will be used in gathering focus group participants?
Data collection schedule	What is the timeframe for the data collection? When will it begin and within what period of time will it be completed?
Data recording method	How will the data be recorded? Sample answers: on preformatted observation sheets, on an Internet survey form
Quality control	How will the data be kept confidential? How will errors and bias be identified and reported?

It's helpful to document the data collection in a summary record. You may have to create a more fully developed plan if your organization requires it, or if great amounts of data will be collected, but even a simple summary document is a good checklist. It's a high-level plan and should be kept in your strategic planning notebook. Example 3.1 illustrates a data collection summary record.

Choosing Data Collection Methods

There are several different methods you can use to gather data, and more than one of them can be used to select the same data. Understanding the various methods and their respective advantages and? disadvantages will help you make better decisions about how best to collect the data you need. You will want to consider such things as

EXAMPLE 3.1

High-Level Data Collection Summary Record

  availability of resources

  quality and quantity of available information

  cost of development and data collection methods

  availability and consistency of technology and other data-gathering methods across locations

  location of the data (Internet, people, library, a service or consulting agency, and so forth)

  characteristics of your sample (For example, if people form your sample, what is their ability to access and complete surveys or be involved in focus groups?)

  data source (some resources are better for certain topics than others)

  collection schedule (that is, how time sensitive your data collection is)

  required response rate (For example, if you're using a survey or a focus group, how many responses do you need per question to establish data validity?)

  burden on respondent (that is, the degree to which the response depends on his or her effort)

  complexity (that is, how deep or detailed the data must be)

  possibility of bias introduced by the person doing the data collection

  control of the data (that is, how sure you are that the data are from the desired source).

The most common methods of data collection used in strategic planning are listed in table 3.1, and evaluated relative to the considerations listed above. Using this table to compare the advantages and disadvantages of the methods against your data requirements will help you quickly decide which method(s) to use.

There are other methods that are more technology driven and generally more automated. Because these methods usually are unbiased, the data mined from them are extremely valuable. One word of caution—if these methods are not currently available in your organization, the cost of implementing them solely for the purpose of gathering information for strategic planning may be prohibitive. Table 3.2 lists some of these methods and explains how they can be used in data collection efforts.

TABLE 3.1

Considerations for Selecting Common Data Collection Methods

When you've identified the needed data and the optimum methods for gathering them, you're ready to develop your collection tools. Let's get a more detailed look at focus groups, interviews, surveys, and background research.

Focus Groups and Interviews

Specific groups of people—business partners, current or prospective customers, or competitors—have information that you need and that you can't get elsewhere. Focus groups or interviews work best with specific individuals who have valuable information (for example, the business managers from five organizations in your industry) or with a small and narrowly defined group (for example,5–15 randomly chosen customers who fit a specific demographic profile).

TABLE 3.2

Technology-Driven Data Collection Methods

Method	Data Collection Use
Weblogs, or blogs	Blogs can be used to research an idea or ask a question and gain feedback. For example, if you set up a blog for your field sales people to report how they are using their computers, they can give you that information and/or make suggestions about what other computer programs and materials might assist them in selling.
Kiosks, touch-sensitive screens	By placing kiosks in malls, recreation areas, grocery stores, or other appropriate venues, you can conduct spontaneous surveys, or you can provide specific information to a person who then can react on the spot. For example, asking a question such as, Would you buy a printer today if it was on sale for $250? will elicit potential customer responses to product pricing. Product features, pricing, and many other types of information can be gathered with touch-sensitive response screens.
Optical mark recognition	Written characters can be scanned into electronic data collection vehicles so that information from hand-written customer surveys can be collated digitally.
Intelligent character recognition (ICR)	After data collectors use paper-based forms with their interviewees, the ICR scans the forms and enters the data into an electronic database that can be downloaded as needed.
Bar codes	Whereas bar codes on products in stores help manage inventory, they also can be used to determine what types of customers purchase certain products and at what prices. The capture of product and pricing data for each customer enables a company to know where and when products are being purchased and by whom.
Data capture	Cookies (small data files created by a Web server and stored on your computer to allow a Website to recognize you and keep track of your preferences) and electronic tracking systems operate in the background to capture data from personal and organizational computers. Within an organization, data capture lets you see, for example, how specific data are accessed or how often product information is used in a sale. This technology also can be used to gather data from prospective customers as they complete Web-based forms before gaining access to your Website. And it can be used for online surveys.

Whether you conduct interviews or focus groups depends on several factors. Table 3.3 discusses the differences between the two methods.

The mechanics for both methods are very similar. The first task is to list your objectives for the focus group or interview. Then select the sample of respondents by determining the participant characteristics you will need to meet your objectives. You may want to target specific interviewees (for example, senior managers, prospects for new products, current customers), or you can use sampling techniques (such as random or stratified sampling) to select participants. You'll find sampling tools for this purpose on the Internet, and here are some good sources to consult:

TABLE 3.3

Comparison of Focus Groups and Interviews

Focus Group Characteristics	Interview Characteristics
Use planned discussion in a group to gain perceptions on a defined area of content.	Use planned discussion one-on-one to gain perceptions on a defined area of content.
Environment is nonthreatening.	Environment is nonthreatening.
Participants share ideas and perceptions and influence each other by piggybacking on ideas.	Participants share ideas and perceptions in a private conversation.
The group hears and responds to ideas.	The interviewer is a passive participant, and conversation is very confidential.
The interviewer is not usually in a directive role.	The interviewer is usually in a directive role.
More people can be interviewed in a shorter timeframe.	Method is time consuming if more than three or four people are to be interviewed.

  Creative Research Systems, www.surveysystem.com

  Survey Guy, www.surveyguy.com

  Custominsight, www.custominsight.com/articles/randomsample-calulator.

The third and perhaps most challenging step to conducting a focus group or interview is to develop the guide (or script). Your focus groups and interviews should be as consistent as possible so the data you gather from them is more easily analyzed. To ensure that the questions or order are not confusing or leading, you should develop a guide to keep the discussion on track. The guide will contain an opening statement (usually the objective of the research), the questions (with follow-up probes), and the closing remarks.

In developing an interview or focus group guide, you first need to answer the following questions:

  What do you want to know as a result of the interview or focus group?

  How will you sequence the questions for the interview or focus group?

  How much detail do you wish to solicit from the interviewee or participants?

  How long do you think the interview or focus group should last?

  How should you word the questions?

Review examples of interview or focus group scripts with objectives that are similar to yours. Two sites on the Internet offer examples of interview and focus group scripts and additional resources: Flashlight Evaluation Handbook (www.learningobjects.wesleyan.edu/downloads/focus_groups_protocol.htm) and Office of Research Ethics (www.research.uwaterloo.ca/ethics/human/application/101samples.htm). Identify what was successful about those scripts and, if problems existed, try to determine why. Then duplicate the successes.

Although interviews and focus groups usually use qualitative questions, you can gather some quantitative data using questions with prespecified multiple-choice or ranking responses (such as high/medium/low or always/sometimes/never). Here's an example of a question with prespecified multiple-choice responses:

If you were given a choice of spending a $500 bonus on one of the four items listed, what would you spend it on: (1) technology, (2) travel, (3) education, or (4) entertainment?

Here's an example of an open-ended question seeking essentially the same information but with no limiting factors included:

If you were given a $500 bonus and had a choice about how to spend it, what would you buy?

Tool 3.2 defines and exemplifies six question types that are most commonly used in interviews and focus groups.

Typically you will design the interview or focus group questions to begin with less-sensitive or more easily answered questions, such as demographics or background. (Remember that demographic questions can quickly become tedious, especially in a focus group, so try to limit them.) It's usually easier to ask experience questions first and address opinion questions later after you have established a level of trust.

TOOL 3.2

Six Commonly Used Types of Focus Group or Interview Questions

Type of Question	Definition	Examples
Background or demographics	Asks identifying characteristics of those interviewed (typically education, role, business unit)	What is your role at XYZ Corporation?   Do you have children at home who are under 18?
Experience	Seeks details of a situation or experience	Tell me about the sales training you have had in the past.   How often do you use [product]?
Sensory	Questions what has been seen, heard, felt, or touched	When you walk into one of our stores, what is the first thing you see?
Knowledge	Assesses what information or knowledge is present	Describe for me what I would hear the customer service representative say to a customer calling about a product issue.   What circumstances enable a customer to be eligible for fee waivers?   What are the procedures for dealing with a safety hazard in our organization?
Feeling	Determines how the interviewee feels about a topic	What is your confidence level when making a customer call on investment portfolios?   To what extent do you feel concerned when providing a credit card number to a company online?
Opinion or value	Determines what the interviewee thinks about an issue	Did the automated phone system make the customer service experience more satisfactory?   What product changes would you recommend?

One of the most important things to remember when writing questions for interviews or focus groups is that each question should focus on a single idea. The question, What are our key business drivers, and how do you think we can overcome the issues they create? contains too many elements and will hinder meaningful discussion. Asking the two parts of this question as separate questions will be more effective. The number of why questions you include in the guide should be limited because they tend to irritate interviewees or participants when they are asked repeatedly.

When you've developed the questions, three activities remain:

1. Pilot test the process with a few members of the target population using a prototype or preliminary version of the guide. Identify any problems, such as misunderstood questions, and revise the script to fix them.

2. Conduct the interviews or focus groups, and track the responses using a previously agreed method to ensure consistency and make analysis easier later on. Whereas one person is needed to conduct interviews, two people from your team will be required for a focus group—one to facilitate and one to act as scribe.

3. Compile the notes for data analysis.

Example 3.2 is a guide used in interviews with stakeholders. The guide defines the purpose of each question and provides suggested probes for digging deeper during the interview.

Surveys

Surveys offer confidentiality to respondents and can be administered to large groups, typically a customer group or an employee base. They also are a good method for collecting data from respondents in a variety of locations. After you've identified the need for and objectives of a survey, your next actions are

  setting a budget and a schedule

  identifying the sample size and any participation criteria (for example, demographics)

  deciding if you need help in designing the survey (for ex-ample, technical expertise, subject-matter expertise)

  designing the survey

  testing the survey

  implementing the survey

  coding the completed surveys or entering the results into a database.

Here are few tips for successful surveying:

  Keep the survey as short as possible. If you can make it one page, do so; two pages is the maximum desirable size.

  Provide a small reward either for everyone who returns the survey or for a limited number of respondents. (If possible, make that number the amount of responses you need for the survey sample size to be valid.)

  Try to make distribution and collection as time efficient as possible. Email is extremely efficient, especially for internal surveys.

EXAMPLE 3.2

Interview Guide

Introductory Remarks: [Start by introducing yourself.] This interview is being conducted with you and others in the organization. As you know, we are in the process of developing and executing [or revising] a strategic plan. To ensure that the final plan is closely linked with our business and that the products and services we offer will continue to support our customers and employees, we have developed a 10-question interview. As I told you when I made this appointment, I expect this interview to take no longer than 45 minutes. Do you have any questions for me before we begin?

Question	Purpose	Suggested Probes
1. What do you see as our organization's mission?	To help understand how the mission is seen or how it needs to be revised	What is the core reason our organization exists?   Is that reflected in the mission?   Could you put that in one to three sentences?
2. How do you feel this mission aligns with the way our organization works?	To uncover any disconnects between the mission and the “real” organization; to help link the mission to the organization	What is missing in our mission statement?   Do you think we “walk the talk” relative to the mission statement?   How should the mission statement change?
3. What key challenges face our organization in the future?	To ensure that the outcomes have tactics that will meet these challenges	What barriers exist to our organization's success (externally, internally)?   What about this challenge is unique to our organization?
4. What factors in the business environment are forcing change in our organization?	To identify and validate business drivers	What business drivers are forcing the business to change (for example, increased competition, lack of resources, decreased revenue)?   How do these factors/drivers affect our organization?   What or who is responsible for these drivers?
5. What do you think is the biggest risk to our organization in the future?	To uncover organizational vulnerabilities	What internal or external influences pose a threat to the organization? Why?   How do these risks affect our organization?
6. In what ways do you think our products and services could provide more value to our customers?	To understand how others feel the organization can improve its customer link	What is the value of our products and services to the customer?   How could this be increased?
7. What are our organization's core strengths?	To determine what others think the organization does best	What do you think we do best as an organization?   How do you know that?   What do you think is important about these strengths?
8. What do you think is the biggest human resource challenge we face today?	To determine what others think needs to happen for employees	What is challenging about this?   In what ways can we meet this challenge?
9. What do you think is the biggest technological challenge we face today?	To determine what others think needs to happen with regard to technology	What is challenging about this?   In what ways can we meet this challenge?
10. What do you think is the biggest competitive challenge we face today?	To gain a perspective on what the organization needs to do to be more competitive	What is challenging about this?   In what ways can we meet this challenge?

You should schedule one or two person-days to create and test your survey. Survey writing takes time. First you must identify what you want to ask and then you write and rewrite the questions to ensure they are clear, concise, and not misleading. Tool 3.3 provides guidelines for developing survey questions, along with examples of survey items that are poorly written and items that are better.

Before testing your survey, you may want to compare it against the standard criteria for effective questions listed in Worksheet 3.2.

A sample survey is presented in example 3.3. This survey was used to identify management styles existing in the sales department.

The Internet is another tool that is helpful in both survey development and survey delivery. There are several services that will help you deliver surveys across the Web (and some actually will do the design). This is especially helpful if you're trying to do a blind survey with customers or prospects, and don't want any preconceived ideas to form because the respondents relate the survey to your organization. Here are some of these services:

  ZapSurvey,www.zapsurvey.com

  Zoomerang, www.zoomerang.com

  Vanguard Software Corporation, Vista Online Surveys, www.vista-survey.com

  Nooro Online Research, www.nooro.com.

Background Research

Background research is best used to gather information on trends, the marketplace, competitors, or other critical topics. Much of this information is readily available. The conclusions and recommendations for your strategy's tactics most likely will be based on the information and themes revealed by this research.

The biggest problem with background research is spending too much time obtaining unnecessary data. Setting objectives and being realistic about what you want to know and the sources you are exploring can help you avoid this problem. Effective background research begins with these three activities:

TOOL 3.3

Guidelines for Developing Survey Questions

Guideline	Poorly Written Items	Improved Items
Use wording that is simple and to the point.	Do you remember when you last took a class from the training and development department? Was it less than six months ago?	Have you taken a class from training and development in the last six months?
Use wording that is neutral and does not bias the respondent.	Some think that only classroom delivery is effective for training and nothing else works. What do you think?	Which of the following alternative delivery programs would you rather participate in? [Delivery programs are listed.]
Be careful not to influence respondents when providing explanations.	What are your top three priorities for employee development this year (for example, leadership training, technical training, sales training)?	Training and development is working on its program delivery plan for the next year. To ensure that we meet your needs, list your top three priorities for employee development this year.
Provide response intervals that will give you meaningful information.	How many employees report to you? ——1 ——5 ——10 ——15	How many employees report to you? ——Less than 5 ——6-10 ——11-15 ——More than 15
When using prespecified responses to a question, provide no more than 10 responses to avoid confusion.	Which of the following classes have you taken in the past six months? [All 20 classes are listed.]	Which of the following management classes have you taken in the past six months? [Six management classes are listed, with None of the above included.]
The alternative responses should be logically ordered if using intervals, or randomly ordered if using a list of responses.	How would you rate your experience with paper-based self-study for interview training in the past? ——OK ——Poor ——Great	Select the delivery methods you would prefer for interview training: ——Web seminar ——Classroom ——Paper-based self-study
When using a Likert scale, define five, seven, or nine points to avoid confusion.	Please rate the opportunities you have to gain increased responsibility in your job, on a scale of 1 to 7, where 1 is No opportunities and 7 is Often given opportunities.	Please rate the opportunities you have to gain increased responsibility in your job: ——1—Never offered ——2—Rarely offered ——3—Seldom offered ——4—Sometimes offered ——5—Regularly offered

WORKSHEET 3.2

Criteria for Judging the Effectiveness of Your Survey Questions

Criterion	Does Question Meet Criterion?
The respondents understand the questions (the pilot-testing should reveal this).	Yes    No
If response choices are used, they are clear, and they elicit the desired information.	Yes    No
The questions and responses are comprehensive, and they cover a reasonably complete range of answers.	Yes    No
The questions motivate responses for all of the information that you have identified as important. There is no redundancy, and nothing is missing.	Yes    No
The survey is an acceptable length.	Yes    No
The questions honor the participants' privacy.	Yes    No
The survey avoids asking participants for information that is gathered another way. For example, if the employee number indicates his or her division, the survey should not ask for the employee's division location.	Yes    No
The level of wording is acceptable for the audience (for example, highly technical language is not used for novices; language level does not exceed grade 8 for a general population).	Yes    No

1. Creating questions that define the research. For example, what are the parameters of the search? (Go back to your outcomes, goals, and objectives for assistance with this.) In the second case example we offered at the beginning of this step, the parameters were best practices in alternatives to classroom delivery.

EXAMPLE 3.3

A Survey of Management Styles

EXAMPLE 3.4

Research Plan Based on Research Objectives

2. Identifying the resources or sources you will use. Associations or industry groups, publications, other companies, the Internet, and technical libraries are good possibilities.

3. Identifying your research method. Examples include a literature search and benchmarking an organization.

Examples 3.4 and 3.5 illustrate, respectively, a research plan based on research objectives and a plan based on research questions. Document the questions or objectives for each of your data gathering, the sources you'll use, the data collection method, and a timeframe for completing the data gathering.

Tool 3.4 can help you target the background information you need for your plan and the best sources for gathering it.

Here are some helpful tips for your data search:

  Identify all the key words that define or relate to the topic you're researching—for example, quality, total quality management, Deming, team management. Use these terms in Web search engines and book indexes to locate information on your topic.

  Identify all the companies that are successful at this type of program, are considered your competition, or have best practices you would like to learn more about.

EXAMPLE 3.5

Research Plan Based on Research Questions

Research Definition Question	Planned Source	Research Method
1. What in South America makes the most sense for our expansion?	Multilateral Investment Fund   National Political Infrastructure and Foreign Direct Investment   World Advertising Research Center (www.warc.com)	Literature search
2. What approaches to human resource benefits is our competition planning for the future?	First Research   InfoScouts (www.infoscouts.com)   Dun & Bradstreet	Mixed approaches
3. What expected barriers or obstacles are there in introducing new technology to our customer?	Customer demographics   Benchmarking Exchange (www.benchnet.com)	Benchmarking
4. What are the trends in marketing to new prospects?	Marketing organizations/ associations   NetSuite (www.netsuite.com)	Internal marketing study
5. What are the most common quality issues in introducing new technology?	Quality organizations/associations   Tunu Pure Web Searching (www.tunu.com)	Mixed approaches
6. What are today's best practices in the industry?	The Hackett Group (www.thehackettgroup.com)   Benchmarking & Best Practices Council	Benchmarking

  Complete an industry search in your own industry or an industry known to be successful in this type of program, or both.

TOOL 3.4

Sources for Targeting the Background Information You Need

Information Needed	Rationale	Data Sources
Trend information	To plan for the future or to target goals for professional practices	Internet   Best practices organizations   Professional associations
Competition practices	To determine what your competition is doing that perhaps you should (or should not) be doing	Internet   Periodicals   Professional associations
Customer information	To identify what your customers want or need and how they prefer to receive information or training	Customer or customer group
Business partner information	To identify what business partners are doing, what their plans are, and how you can support them	Business partner
Specific requirements for a program or technology	To identify what is needed to support a certification program, to do testing, to implement e-learning, to do a results evaluation, and so forth	Internet   Periodicals   Professional associations   Best practices organizations   Expert in area of requirements
Benchmarking or best practices information	To identify what is considered a best practice or to gather baseline information for comparison with a group doing the same thing	Internet   Periodicals   Professional associations   Best practices organizations   Benchmarking groups

  Complete a search by type of periodical or service—for example, a search of Harvard Business Review for like topics.

  Conduct a search using your business drivers as search terms.

Identifying Preexisting Information

Frequently you can save time in Step 3 by identifying the data that already exist and the source of that data. If you're trying to pin down specific information within your organization, try the following resources:

  internal communications (for example, newsletters, event management, internal communications)

  marketing

  HR recruitment

  directory or information services

  quality improvement department.

Types of data typically found within an organization or externally from research resources are listed in table 3.4. The first column lists the types of information you may be seeking, and the second column lists common sources of that type of data. You can use this table to jump-start the gathering of preexisting information and to save you time and other resources. Although not exhaustive, this list may help you identify where specific information is located.

Maximizing Validity and Reliability

Whether you're doing an interview, running a focus group, sending out a survey, or doing background research, conducting a trial is the best way to ensure that your data are valid (or sound) and reliable (or dependable). By conducting a trial you will ensure that the audience you have chosen is not biased and has the characteristics you need to get the data you require. A trial also ensures that the instrument or method you're using will provide the data you want. In this way you can find out beforehand if the key search term you used located data you didn't need or if a survey question gathers unusable data.

TABLE 3.4

Potential Sources of Preexisting Data

Type of Information	Potential Sources
Industry norms, trends, competitive information	Industry associations (generally can be located via the Internet)
Training norms, trends, research information	International Society for Performance Improvement (www.ispi.org)   American Society for Training & Development (www.astd.org)   Training SuperSite (www.trainingsupersite.com)   Brandon Hall (www.brandon-hall.com)   The Masie Center (www.masie.com)
HR norms, trends, and research information	Hr-esource.com (www.hr-esource.com)   Hrfree.com (www.hrfree.com)   Human Resource Development (www.humanresourcedevelopment.start4all.com)   Society for Human Resource Management (www.shrm.org)
Best practices, bench-marking, and the like	American Productivity and Quality Center (www.apqc.org)   American Society for Training & Development Benchmarking Forum (www.astd.org)   Benchmarking Exchange (www.benchnet.com)   Benchmarking & Best Practices Council   Electronic Frontier Foundation (www.eff.org)   Gantthead (www.gantthead.com)   International Stormwater Best Management Practices Database (www.bmpdatabase.org)   The Data Administration Newsletter (www.tdan.com)   www.pearsonncs.com   LexisNexis (www.lexisnexis.com)   Business.com (www.business.com)
General research	ERIC Clearinghouse (www.ericacve.org)   Conference Board Research Publications (www.conference-board.org)   International Data Corporation (www.idc.org)   Google search engine (www.google.com)   Ask search engine (www.ask.com)
Customer data	Sales or marketing departments
Organizational financial results and history	Shareholder communications or investor relations department (sometimes part of the finance or accounting department; may also be in marketing)
Processes and procedures	Internal communications, technical writing department
Industry data and global information	Global Insight (www.globalinsight.com)   Elliott Wave International (www.elliottwave.com)   Industry Insight (www.industryinsight.co.za)
Financial forecasting	Kiplinger (www.kiplinger.com)   Business.com (www.business.com)   Elsevier Publishing (www.elsevier.com)   Commerce-Database Business Directory (www.commerce-database.com)   Lonee Corporation (www.lonee.com)   Hoover's, Inc. (www.hoovers.com)   Financial Management Association (www.fma.org)   Solution Matrix (www.soluionmatrix.com)   American Institute of Certified Public Accountants (www.aicpa.org)   Internet Public Library (www.ipl.org)   Association for Financial Professionals (www.afponline.org)   Ohio State Fisher College of Business (www.fisher.osu.edu/fin/journal/ofsites.htm)
Marketing associations	American Marketing Association (www.marketingpower.com)   Direct Marketing Association (www.the-dma.org)   Marketing General Inc. (www.marketinggeneral.com)   Produce Marketing Association (www.pma.com)   Business Marketing Association (www.marketing.org)   eMarketing Association (www.emarketingassociation.com)

Many factors influence the validity of data collection, and some factors can cause more harm than others. These factors include

  biased selection of respondents (for example, selecting employees from one area because you know they will participate)

  poor instrument creation

  the Hawthorne effect—being selected makes participants behave better

  respondent bias, whereby respondents want to influence you to do what they want done.

One other factor is worth noting: lack of contrary data collection. Typically there is some form of unconscious motivation or intention that will shape the questions we ask to create the outcomes we believe will be the case. To avoid this, it is important to ask for contrary data collection or to gather information from a view that is different from your own.

Being aware of potential validity and reliability problems is the first step in controlling them. As you move toward data analysis, identify potential problems that may have occurred as you collected the data. Knowing where your data are vulnerable and admitting it tends to stop arguments or disagreements when gaining commitment to your tactics. Table 3.5 identifies potential quality problems and explains how to avoid them.

TABLE 3.5

Methods for Controlling Data Quality

Quality Problem	How to Avoid the Problem
Lack of adequate experimental control	If modeling, testing, or some form of experimental de-sign is part of the data gathering, it's important to en-sure that the experiment has controls in place. These controls should not be manipulated. There must be specific timeframes, protocols for how and when groups are measured, clear guidelines on the extent to which the variable will be manipulated by the research, and random assignment of individuals to their experiment groups.
Inappropriate sample sizes	An incorrect sample size is often used, either because it is difficult to get the required number of responses or because there is not enough time to gather them. Conclusions can be incorrectly formulated because the sample size was too small to identify differences. It is best to choose the methods of data collection based on your sample size. Suggested sample sizes and data collection methods can be accessed through the Websites listed in this step.
Failure to see other solutions	While collecting and analyzing the data, it is important to keep an open mind and track other opportunities with vigilance. Oftentimes the analysis of one set of data will lead you to other information that you had not planned to collect. To make sure you have collected the right information, you must be willing to explore all “information avenues.”
Failure to preserve data	Preserve the original data so that further or independent data analysis can be done, if desired. Keep all data, electronic or paper-based, until you have completed the strategic plan. If questions arise, you may need the raw data to support a conclusion, decision, or recommendation.
Misclassification or inaccurate structuring of the data	Precoding data collection forms for surveys and interviews can help in later data analysis problems. This allows the data to be structured (and thereby entered) correctly.
Poorly developed instruments	Instruments (interviews, surveys, observation forms) that are poorly developed will create quality issues. If the respondent is unsure what to do or has several choices, the data will be a comparison of apples and oranges and thereby invalid.
Lack of confidentiality	Ensuring that sensitive data are collected confidentially and maintained in the same manner is key to others believing in the data and their truth. If specific information is attributed to a person or a group, you run the risk of undermining your credibility and that of your plan.

With the pertinent data in hand, the strategic planning group is ready to begin its analysis. That is the work of Step 4.