Chapter 8. Getting Good Samples and Data
What you will learn in this chapter is how to take good samples and get good data. Otherwise, the best thinking won't matter much because you won't be able to put it to the test properly. In later chapters you will see how to calculate minimum sample size and how to verify that a gauge used to measure a product is giving data that are sufficient for your needs. Just as important, however, is making sure that your sample and data truly represent the population of the process you wish to measure. The whole intent of sampling is to be able to analyze a process or a population and get valid results without measuring every part or component, so sampling details are extremely important.
NOTE
Issues in Getting Good Data
Manufacturing Samples and the resultant data have to represent the total population, yet processes controlling the population are often changing dramatically, due to people, shift, environment, equipment, etc.
Sales Sales forecasts often use sampling techniques in their predictions. Yet the total market may have many diverse groups to sample. These groups may be affected by many external drivers, like the economy.
Marketing What data should be used to judge a marketing campaign's effectiveness, since so many other factors are changing at the same time?
Software Development What are the main causes of software crashes and how would you get data to measure the "crash-resistance" of competing software?
Receivables How would you get good data on the effectiveness of a program intended to reduce overdue receivables, when factors like the economy exert a strong influence and change frequently?
Insurance How can data measuring the satisfaction with different insurance programs be compared when people covered by the programs are not identical?
We have all seen the problems pollsters have had in predicting election outcomes based on sampling. In general, the problem has not been in the statistical analysis or in the sample size. The problem has been picking a group of people to sample who truly represent the electorate!
The problem of sampling and getting good data has several key components. First, the people and the methods used for taking the samples and data affect the randomness and accuracy of both. Second, the product population is diverse and often changing, sometimes quite radically. These changes occur over time and can be affected by location. To truly reflect a population, anyone sampling and using data must be aware of all these variables and somehow get valid data despite them.
I will share some of the difficulties or challenges my teams have experienced in getting representative samples and data. I will then discuss some approaches to get useful and valid data despite these issues. Most of the examples pertain to manufacturing, but I will explain later how the approach recommended for getting good data applies to many other applications.