6.2.1. Descriptive analysis: learning from past behavior to influence future outcomes

In its simplest form, data analysis involves a form of descriptive analysis (Delen and Demirkan 2013). Descriptive analysis is simply the analysis of historical (past) data of an event to understand and assess its trends over time. It involves the use of analytical techniques to locate relevant data and identify notable trends in order to better describe and understand what is happening in the data-set (Sedkaoui 2018b).

These techniques are therefore used to describe the past, which can refer to any time in which any event has occurred. Consider basic calculations such as sums, means, percentages, etc., as well as graphical presentations. This analysis is based on standard database functions, which just require a knowledge of some elementary calculations.

The most important thing is that the company can have a clear vision of the future results of the event (production, operations, sales, stock, customers, etc.), based on how it has operated in the past, in order to know how it should act. This provides a clear explanation of the behavior of an event and why certain events occurred. The objective is to find the reason for success or failure.

What makes this type useful is the fact that it is possible to draw lessons from an event, based on its behavior in the past. This helps to understand how it can influence the future. Of course, by using the right set of tools, the company can learn powerful lessons.

This type of analysis is of great importance and uses predictive models that are considered a subset of data science (Waller and Fawcett 2013; Hazen et al. 2014).

6.2.2. Predictive analysis: analyzing data to predict future outcomes

With the increasing amount of data, the improvement of computing power, the development of Machine Learning algorithms and the use of advanced analysis tools, many companies can now use predictive analysis. This analysis goes beyond the previous analysis, it aims to predict future trends.

It should be noted here that the word “predict” is not synonymous with what will really happen in the future. This cannot be the case and no analysis or algorithm can do it flawlessly. Indeed, predictive analysis can only predict what may happen in the future. The basis for this type of analysis relies on probabilities.

By providing usable information, predictive analytics allows companies to understand the future. This type of analysis involves predicting future events and behaviors in data-sets, based on a model constructed from similar previous data (Nyce 2007; Shmueli and Koppius 2011).

There is a wide range of applications in different fields, such as finance, education, health and law (SAS 2017; Sedkaoui 2018a). From analyzing sales trends based on customers’ buying habits (to recommend personalized goods or services), to forecasting demand for operations or determining risk profiles for finance, in addition to analyzing feelings (otherwise known as a sentiment analysis) etc., the scope of this analysis is vast and varied.

Predictive analysis can be considered one of the most commonly used methods by companies to analyze large data-sets to find an answer to the following question: what could happen based on previous trends or patterns?

To answer this question, predictive analysis combines all the data, filling in the missing data with historical data from company databases, and looks for models that identify the relationships between the different variables.

In this case, it should be mentioned that the amount of data available is not a problem; the wealth of the data is, however, often questionable (Sedkaoui 2018b). This is necessary when people want to perform a prescriptive analysis.

6.2.3. Prescriptive analysis: recommending one or more action plan(s)

Delen and Demirkan (2013) found that Big Data has introduced the possibility of a third type of analysis, called “prescriptive analysis”, which goes beyond previous analyses by recommending one or more action plans to be undertaken.

Just as the two types of analysis described above are closely related, prescriptive analysis is closely linked to predictive analysis. Prescriptive analysis uses evidence-based predictions to inform and suggest a set of actions, because in order to be able to prescribe a series of actions, it is first necessary to anticipate the situation in the future.

This analysis not only allows companies to consider the future of their own processes and opportunities, but also to determine the best course of action to take in order to achieve competitive advantages. This means that this type of analysis is more heavily based on determining what actions need to be taken in order to optimize certain results, than on what can happen if the company continues to do the same.

It is an advanced analysis concept based on: optimizing analysis results and identifying the best possible action plans. This type uses a combination of Machine Learning techniques, tools and algorithms.

It thus requires a predictive model with two additional components:

• – the data;
• – the evaluation system, or feedback, to analyze and monitor the results of the measures taken.

Prescriptive analysis can influence the company’s activities and even the way decisions are made. It also has a very significant impact on companies in all sectors by enabling them to improve their effectiveness and become more efficient. Prescriptive analysis is therefore used to quantify the effect of future decisions in order to indicate possible outcomes before decisions are actually made.

The autonomous car is the best example of the application of this analysis. This car analyses and assesses its environment, and decides on the action to be taken based on the data. It can accelerate speed, slow down, change direction to avoid road traffic, etc. But it should be noted that this car also relies on predictive and descriptive analysis in its data analysis process.

So, with one of the three analyses previously described, or by combining two or three at a time, a company can understand its data and generate relevant information at different levels of analysis, that can facilitate decision-making and inform the different action plans.

Leveraging data to analyze trends, creating predictive models to identify potential challenges and opportunities in the near future, these all offer new ways to optimize processes and improve performance. Different types of analysis are established by experts in the field in order to facilitate the interpretation of the data.

However, the company must ensure it chooses the right analytical option in order to increase its ROI, reduce its costs, operationalize its activities and guarantee its success.

The easiest way to do this is to look at the answers that each type can generate.

6.2.4. From descriptive analysis to prescriptive analysis: an example

From one type to another, as can be seen, the analysis must be applied in a sequential way, in other words, descriptive, then predictive, then prescriptive. However, this does not mean that one type is better than another, on the contrary: these three types complement each other (Sedkaoui and Khelfaoui 2019).

What? Were the previous discussions not clear enough? Still can’t tell the difference between these three types of analysis? Do you want to understand what these different types of analysis mean? And what type of analysis is most appropriate and for which situations?

You probably need some additional explanations. In this case, we will use a simple example, hoping that it will help you.

Imagine that you are a mobile application developer, and you want to get an overview of your business. First, you would start with a simple analysis to calculate the number of downloads of your applications and the profits you have made over the last three years, for example. You would also, for example, define certain cohorts of interests, such as the most important applications downloaded, the least downloaded applications, etc., and calculate the benefits of each.

This is simple, because it is just an assessment of your activity, and it is done in order to understand its behavior (its history). What you have just done is describe your activity based on the data you already have. This is the descriptive analysis.

Now let’s assume that you want to develop these applications to expand your business (create new applications or develop some of them). The problem is that you can’t have accurate information about which applications will be the most popular in the future or how much profit they will make, etc., because all you have is historical data about your business.

What you can do here, for example, is take this historical data and create a model that will allow you to predict what will happen in a month, in six months, in a year or more. So, depending on the current and past situation, you will have a more advanced forecast (such as the number of downloads, profits, etc.) on your activity. You have then moved on to the second level of analysis, or predictive analysis.

Now, let’s imagine that you want to go further in your business by developing a new application for e-learning and increase its benefits, and that you want to know what you need to do in order to achieve it. Would you need to look for new customers (universities, etc.) (Plan A), stop developing the least downloaded applications (Plan B), develop relationships with the academic world or professionals who are best qualified to understand learning needs (Plan C) or launch an advertising campaign (Plan D)?

To know which plan to choose, you would need to illustrate some forecasts. But maybe you’ve never done an advertising campaign and you’ve always worked with the same customers. In this case, you must use new data sources to calculate the strategic effects of actions A and D. You would also need to look for an optimal solution that combines actions A, B, C and D. This is where the third type of analysis comes in, which is called prescriptive (or optimal) analysis.

6.3.1. Definition of objectives and requirements

The first step in each process is to define the objectives that need to be achieved. It means asking the right questions. The aim here is to make sure that you have clearly defined the what, the how and the why, in other words, the context of your project, so as to hold on to the best action plan.

This means that before you enter the data analysis phase, you need to understand the context in order to define the main problems and identify needs.

At the end of this small exploration step, you will have a global view of what you want to accomplish. This step must take place before there is any data.

This step must be done before because once you have collected the necessary data, you will examine it to guide your decision-making strategy. At this point, you will need a baseline or indicator to determine whether the project will achieve its objectives.

This reference or indicator should be defined at the beginning of the process. This means defining the target by considering the different constraints and potential solutions that will make it possible to achieve this target.

Let’s keep to the previous example: if you want, for example, to develop an application, it will be necessary to define the notion of “interest” that lies behind the creation of such an application, and then produce the work plan in accordance with the requirement of that interest. It is important here to understand the motivation of your project before undertaking the actual analysis tasks.

This step will therefore allow you to explore all possible avenues in order to identify the different variables that directly or indirectly affect the phenomenon that interests you. This will help you understand the following:

• – the objective (target);
• – the context (opportunities and challenges);
• – data sources;
• – the necessary analytical techniques;
• – relevant technologies;
• – the cost;
• – the necessary time.

6.3.2. Data collection

Before you begin the data collection step, you must understand and identify the data that could be useful for your business. It is not just about the quantities of data or Big Data, but the value generated by the analysis of these data.

In the era of Big Data, the amount of data produced will continue to grow. There will be more data from various sources in real-time and in different forms, meaning that data will be captured and stored in many forms, ranging from simple files and database tables, to emails, photos, videos, etc. This means that you need two types of data (Sedkaoui 2018b):

• – Those that are already stored internally, or the data you have. These may include databases, various digital files, e-mails, photos, archives, etc. You will search these data sources for the data that are relevant to your project.
• – The ones that come from outside, that you will couple with your internal data. External data refers to all data that is not generated by your activity. Among these data, we can distinguish the following: social network data, videos, tweets, geolocation data, etc. Most of these data are unstructured, which generally makes them difficult to use. But they are useful for enriching internal data.

There is also data from other companies, or even from government and organizations. They make their data accessible to all and share them for reuse. This type of data is known as Open Data. This data can be of excellent quality and depends on the party managing it.

In this step, you will therefore look for the necessary data to achieve the predefined objectives. This will be difficult, because during this phase, you will face some data issues, such as:

• – the volume, variety and velocity, already detailed in Chapter 5 of this book;
• – the complexity of data collected from different sources and in different formats;
• – problems with missing data or outliers;
• – the need to ensure the consistency and quality of data, etc.

You should therefore pay particular attention to this phase, as it is necessary to ensure the reliability of the data before starting an analysis. According to Soldatos (2017), data collection has several particularities compared to the consolidation of traditional data from distributed data sources, such as the need to process heterogeneous data for example.

The data collected in this step will probably be a very important source of information, on which the relevance of the model built will be based. What you need to do now is to prepare this source for the modeling phase. This is extremely important, because the performance of your model depends largely on this step.

6.3.3. Data preparation

Once you have collected the necessary data, proceed to the preparation step. This is the stage that gathers the activities related to the construction of the data to be analyzed and which, of course, have been elaborated from sources (raw data).

During this step, the data will be:

• – classified according to selected criteria;
• – cleaned;
• – coded to make them automatically compatible with the tools that will be used.

It is a real challenge to take advantage of the amounts of data you have been able to collect. You will find that some types of data will be easy to analyze, such as data from relational databases. This is structured data. But if you are working on unstructured data, you need to prepare it for analysis.

Next, these data must be cleaned. Data cleaning is a sub-process of the data analysis process. It corresponds to the transformation of so-called unstructured data into semi-structured or fully structured data, in order to make them more consistent. Data cleaning includes several elements, which are elaborated on in the following three points.

6.3.4. Exploration and interpretation

Once you have collected the right data, to answer your question in the first step, having cleaned and prepared them well, it is time for an exploratory analysis in order to interpret the data and make sense of them. This step will help you understand the composition of your data, its distribution, and the different correlations (looking for similarities and differences) that can result.

Because when you manipulate data, you may find the exact data you need, but it is more than likely that you will have to revise your initial question or collect more data. In any case, this initial analysis of trends, correlations, variations and outliers helps you focus on analyzing your data in order to better answer your question, in addition to any objections that others may have.

One of the simplest ways to explore your data is to use the basic parameters and indicators of descriptive statistics: the mean, standard deviation, frequencies, etc., which give a concise picture of the data and determine the main characteristics, thus allowing you to understand the overall trend.

For example, to manipulate the data, you can use a pivot table in Excel. This table allows you to sort and filter the data according to different variables, perform basic calculations and search for correlations.

You can also present the results in graphical form (histograms, box plots, point clouds, 3D diagrams, etc.) that will be used as tools to visualize the data.

Data visualization occurs when information is presented in a visual form. Its purpose is therefore to communicate information in an understandable way. This makes it easy to compare data when they are presented visually. Cross-referencing data with on another during visualizations allows us to see less obvious relationships to be observed at first glance.

The information will be easier to capture when presented in the form of images or graphs, thanks to the different visualization techniques. It is therefore essential to have an exploratory view during this step, to verify the data and take a step back to correct anomalies if necessary. The objective here is to understand your data and ensure that it is properly cleaned and ready for analysis.

In practice, the techniques described in this step are not limited to visualization techniques or simple basic statistics. Other techniques and methods, such as modeling, clustering, classification, etc., may also be useful for exploratory analysis. However, the statistical and technical context is not enough; you must have a good understanding of the business and be able to recognize whether the results of the models are meaningful and relevant.

6.3.5. Modeling

Now that you have completed the data mining phase and understood the data you have prepared, it is time to move on to the next phase: the model creation phase. This model will be based on a data-set that is representative of the phenomenon (problem) you are trying to model.

This phase is much more specific than the exploratory phase, because you know what you are looking for and what you want to achieve. The construction of the model therefore depends on the available data and the techniques you wish to use.

It should be noted here that when you obtain this model, you will also need to analyze the questions below:

• – does the model really answer the initial questions (objectives) and how?
• – can I easily implement it to explain the phenomenon under study?
• – are there any constraints or considerations that you have not taken into account?

If your model can answer these kinds of questions, then you probably have a model that can reach productive conclusions.

Before making the model operational, you must assess the quality of the model, in other words, its ability to accurately represent your initial problem. This will ensure that it meets the objectives formulated at the beginning of the process. This assessment also contributes to the decision to deploy the model or, if necessary, to improve it. At this stage, the robustness and accuracy of the models obtained are tested.

6.3.6. Deployment

By following the previous steps of the data analysis process, you are making better decisions for your project. At this point, we can say that the choices have been guided and oriented by the data you have collected and prepared, explored and analyzed in a robust way. By performing the various tests, your model becomes more accurate and relevant. This means that you will make informed decisions to manage your business effectively. The emphasis put on the end-use is therefore one of the key factors in the data analysis process.

With clear data in place and a good understanding of the content, you will be able to create one or more models in order to make better forecasts, rank your priorities or better understand the phenomenon you modeled. Once you have acquired it, your model will be ready to be presented. This involves planning the deployment of your model, its monitoring and maintenance.

The deployment can therefore range from the simple generation of a report describing the knowledge acquired, to the implementation of an application allowing the use of the model obtained, for the prediction of unknown values of an element of interest (Sedkaoui 2018b).

At this point, you may think you have reached the end of the process flow, but this is not the case. Because it is during this stage that your general skills will be most useful and extremely important. If you have done things right, you now have an operational model and stakeholders.

This last step involves using the results of your data analysis process to determine the best course of action. This process may seem a little abstract at the moment, but we will deal with all these points in detail in the last part of this book. You will see a great commonality between all these steps.