Using data obviously implies collecting/measuring it first, and then capturing these measurements into a database or spreadsheet. I cannot cover a massive amount of research methodology in this book, so the enthusiastic researcher should also study a complete book on methods. This chapter makes a few critical points about data collection and capture for statistics.

Have another look at Chapter 2, notably the process of statistics diagram. The major data challenges discussed there are choosing the correct samples and populations, choosing the correct constructs and variables, and asking the right questions. I cover each of these topics here. If there is one major point that can be made about getting research methodology right it is this:

There are two cardinal errors that can be made in research; all other methodology issues are secondary to these. The two cardinal errors are measuring the incorrect variables or having an incorrect (wrong or non-representative) sample or set of observations. If your sample is poor then your analysis will not be meaningful to the population you wish it to represent, and you will be unlikely to replicate your findings. If you measure the wrong set of variables then you have missed the focus of the study. Even if you have measured some correct variables but you have an inadequate variable set, your study is being conducted without information in its correct context. We call this specification error. Having good methodology and doing good analysis on the wrong variables or wrong sample is often problematic at best and frequently meaningless.

Having said this, it is also important to do a good job on the other methodology details, such as how you have measured and analyzed data. I make some further comments in the next few sections.