Data Analysis and Data Generation Process

This is perhaps a subtle issue but a good analysis of data can at best provide information about the process that led to the generation of the data itself. If a specific data generation process did not exist then an analysis of the data will provide no clues about the process. For example, consider a situation where a single company or a pool of companies invests in raising customer loyalty levels. In other words, there is no effort or resource expended in reducing loyalty from the current levels or in exploring other options to manage the customer base. Imagine that at some later point an analysis is done to assess whether the strategic choice to try to increase customer loyalty is paying off in financial terms. Now what is important to note is that this analysis, if appropriately conducted, can only tell us whether an increase in loyalty resulted in any change in the marketplace or financial performance of the firm. It will not be able to address issues pertaining to what would have happened if the firm or firms had chosen to reduce loyalty and chosen another strategic path.

Said differently, the data analysis can only inform us about the data generating process. If there is no attempt to choose strategic options that would generate the data relating to those options, one will not be able to learn whether the options that were not chosen would have resulted in outcomes that would have been better or worse than those that were chosen. Therefore, if a firm or a group of firms chooses the path on increased customer loyalty, a subsequent analysis of the data will not be able to uncover whether an alternative path, such as changing the wages of the frontline workers would have resulted in a better or worse outcome. Overall, a firm can discover the links between the chosen paths and observed outcomes but not those between foregone paths and possible outcomes.

Mental models restrict strategic choices to those that make intuitive sense and prevent the discovery of alternative routes to firm profitability. They prevent experimentation with novel paths and tie the firm down to only those that seem to be consistent with the model. Consequently, even under the best case scenario, the firm learns about whether the model-consistent path is worthwhile or not. It fails to gather the information necessary to compare model-consistent and model-inconsistent paths to judge which one would be superior. For example, if a firm follows an innovation-centric mental model and makes choices that are broadly consistent with it, then it may never discover the power of imitation-based strategies.