2.1: Resource management

Several researchers tried to explore applying machine learning algorithms to create predictive models that address a specific problem related to a single site or a country. For example, a research work conducted by Michael LaMantia utilized a triage-based model to create a model that gives a probability of admission rates of elderly patients (LaMantia et al., 2010). This work was performed based on dataset of 4873 visits of patients to the hospital. Triage is the process of sorting and filtering out patients based on priority, it aims to determine a patient’s acuity level in order to facilitate timely and effective care before their condition worsens. The work managed to predict the number of admissions in a total population of visits by elderly patients. Regression modeling is performed to identify the variables that are most significant in predicting the probability of admission. The study concluded that these variables are: age, heart rate, triage score, chief complaint, and diastolic blood pressure. However, this work only applies to elderly patients. This does not provide the broad representation which is needed to properly enable the health administrators to allocate their resources.

Another example is a study published by Weissman et al. (2020) to predict the hospital capacity needs during the COVID-19 pandemic. The study shows that using records of patients with COVID-19 alone, it would be 31–53 days before the demand for more resources exceeds existing hospital capacity. The study used the Hospital Impact Model for Epidemics (CHIME), which is a modified SIR model that is used to compute the number of people infected with COVID-19 in a closed population overtime. The CHIME model estimated that it would be 31–53 days before demand exceeds existing hospital capacity. The study identifies the needed resources in terms of the total capacity for hospital beds, the number of ICU beds and ventilators in the best-case and worst-case scenarios. Such a study is significantly important in identifying the resources that will be needed during a pandemic to help healthcare administrators plan ahead and make the most suitable decisions to optimize their resources.

A third example is the research conducted by Giacomo Grasselli in Italy, which was one of the major spots during the COVID-19 pandemic. Grasselli used historical data to predict the number of patients who will be admitted to the intensive care unit over a period of 2 weeks (Grasselli et al., 2020). This work is prevalent only to that specific hospital and its admissions. The benefit of his methodology is the ability to use real-time data specific to that hospital. The downside, however, is that his forecast does not take into account factors that account for higher infection rates and that these predictions are very region specific.

A final example is a study completed by Chen et al. In their work, authors utilized data from the Tonji Hospital in China and applied five machine learning approaches: logistic regression, partial least-squares regression, elastic net, random forest, and bagged flexible discriminant analysis to select the features and predict outcomes in severe COVID-19 patients (Chen and Liu, 2020). The authors successfully validated their results by using the area under the receiver operating characteristic curve (AUROC), which was applied to compare the model’s performance. They also tested their results on 64 patients admitted to the hospital with COVID-19. The main focus was predicting the number of moralities that will result for the admitted patients. In order to have methodology that properly allows healthcare administrations to allocate resources properly, a prediction of infection rates would be a much more useful feature to work with. Also, focusing on a small hospital in a large country is not enough to give a proper representation of the virus activity. Validation of such methods needs to be established on other subjects not native to the city.

2.2: Impact on people’s health

The second category discusses the research work that used machine learning algorithms to predict COVID-19’s impact on people’s health and well-being.

Before looking at the opportunity to forecast different trends of the pandemic, researchers started to think of what the global impact of the COVID-19 pandemic will be. Answering this question requires accurate prediction of the spread of confirmed cases all over the world as well as analysis of the number of deaths and recoveries in each countries.

Forecasting, however, requires ample and reliable historical data which may be a limiting factor as the virus was fairly new at the time of the study. Moreover, forecasts are influenced by the reliability of the data, vested interests, and what variables are being predicted. Also, the psychological factors play a significant role in how the population perceive and react to the danger caused by the disease and the fear that it may affect them in their own communities. Therefore, the main challenge that faced data scientists at that time is how to predict the impact on people’s health using machine learning algorithms. The risks were significant because the reports generated by these algorithms were used by world leaders to make critical decisions that impact the local as well as international economy.

A study done by Fotios Petropoulos used statistical data from the John Hopkins University and analyzed preexisting graphical representations of trends to attempt and forecast future representations of the graph (Petropoulos and Makridakis, 2020). Petropoulos used exponential smoothing models, which can capture a variety of trend and seasonal forecasting patterns such as mortality, confirmed cases, and recovered cases in 10 day intervals. One of the benefits of this model is that it takes the most recent observations into account and weights them accordingly. For example, if we are looking at 4-month data in 2018, they will likely be weighted differently when considered in the same period of 2020. The exponential smoothing method takes this into account (Avercast, 2020).

In another study completed by Li, where she focused on analyzing the existing data of the Hubei epidemic situation, a corresponding model is then established, and then the simulation was carried out. Through the simulation, she studied the main factors affecting the spread of COVID-19, such as the number of basic regenerations, the incubation period, and the average number of days of cure (Li et al., 2020). This was done using a Gaussian distribution. Gaussian distribution, also known as the normal distribution, is a probability distribution that is symmetric about the mean, showing that data near the mean are more frequent in occurrence than data far from the mean. A normal distribution has two parameters: the mean and the standard deviation. An analysis of graphical representations is useful for short-term expectations of certain data, but it also limits using region-specific data.

2.3: Exit strategies

How did a health crisis translate to an economic crisis? Why did the spread of the COVID-19 bring the global economy to its knees? When will COVID-19 reach its peak? The answer to these questions lies in two methods by which COVID-19 stifled economic activities. First, the spread of the virus encouraged social distancing which led to the shutdown of financial markets, corporate offices, businesses, and events. Second, the exponential rate at which the virus was spreading, and the heightened uncertainty about how bad the situation could get, which led to the closure of airports, a new look at consumption and investment among consumers, investors, and even international trade partners. Preparing to open the economy once again was a very critical decision. Therefore, decision makers needed a trusted methodology to help accurately forecast the peak and termination times of the pandemic. In those circumstances, having such a methodology was an important factor to minimize the risk of further losses whether economically, or physically.

For example, a study conducted by Zahiri utilized statistical data collected from Iran and their published infection rates. The study applied the SIR model to forecast infection rate, peak times, termination times, and other parameters (Zahiri et al., 2020). An SIR model is an epidemiological model that, in this study, computed the theoretical number of people infected with a contagious illness in a closed population over time. The name of this class of models derives from the fact that they involve coupled equations relating the number of susceptible people (S), number of people infected (I), and number of people who have recovered (R) (Smith and Moore, 2004). This model may be useful in interpreting data for a specific country and under specific conditions. However, it was difficult to generalize this model and adopt it in other countries. The disease has thus far been unpredictable and this model did not also account for the wave of infection that can occur if airports and businesses reopen.

Airline business was another major industry that was impacted by the disease. In March 2020, a financial analysis study reported that Canadian airports were confronting the prospect of 2 billion dollars in losses over the next few months as borders shut and planes stay parked due to the COVID-19 pandemic (Times Colonist, 2020). This major loss encourages airport facilities to reopen, to avoid any more losses. But it is important to have methods that enable us to accurately forecast the optimum time to open airports. In our analysis of literature, this aspect seemed to be missing, with the focus primarily being on mortality and infection rates.

4.1: Filter approach

In the filter approach, a ranking-based criterion is used to rank all features in the dataset. Then, a threshold value is set and all features below the threshold are removed (Chandrashekar and Sahin, 2014). The ranking methods involved in this process are known as filter methods as they filter out irrelevant features from the dataset before being evaluated by a classification algorithm. The features that are essential to construct an optimum subset are identified as highly relevant features (Yu and Liu, 2004). It is also important to ensure that there are no redundant features in the dataset to reduce the effect of curse of dimensionality (Chandrashekar and Sahin, 2014). The redundancy of two features can be calculated by calculating the correlation between their values. The following section discusses the major filter-based selection methods.

4.2: Wrapper approach

In the previous section, we discussed the filter methodology which uses an independent measure as an evaluator for subset evaluation. In this section, we discuss wrapper methodology, which uses a learning algorithm as a feature subset evaluator (Kohavi and John, 1997). This methodology uses a learning algorithm as a black box and its performance as an objective function for evaluating a selected feature subset (Chandrashekar and Sahin, 2014; Kohavi and John, 1997).

Various search techniques are used to generate different feature subsets and are evaluated using a machine learning algorithm till the optimal subset is found. These methods perform better at defining the optimal subset that is best suited to a learning algorithm. One of the key advantages of using wrapper-based feature selection approach is that it does not ignore the dependency of the selected feature subset on the overall performance of a learning algorithm (Kohavi and John, 1997). Therefore, the performance of this approach is usually superior. However, since it uses a search technique and a learning algorithm together for the subset selection and evaluation, it tends to be slower and computationally expensive.

In order to optimize the search time, different types of search algorithms have been used in the literature. Various wrapper-based experiments have been carried out by varying different search techniques and learning algorithm as a subset evaluation measure.

4.3: Embedded approach

Embedded methodology performs feature selection as a part of the training process. In comparison to wrapper approach, these methods provide normal or extended functionality to the learning process by lowering the computational cost (Guyon and Elisseeff, 2003; Chandrashekar and Sahin, 2014). During the learning phase of the model construction, they identify the features which will be the best fit for the model based on different independent measures. Following that, these use the learning algorithm to select the final feature subset, which provides the best performance. Decision trees such as C4.5 and random forest are some of the commonly used embedded methods in classification. In regression analysis, embedded methods like Ridge Regression, Elastic Net, and LASSO are used to perform feature weighting based on different regularization models to minimize the outlines and reduce the feature coefficients to be smaller or equal to zero (Tibshirani, 1994; Zou and Hastie, 2005; Yang et al., 2015).

5.1: One-class vs. multiclass classification

OCC or unary classification is different from binary/multiclass classification, as it trains a model with data objects belonging to only one class, called the target class, in the presence of no or limited outlier distribution. The outliers in the data objects are identified by error measurements of the feature values compared to other target class objects. OCC provides a solution for classifying new data by defining a decision boundary around the target class, such that it only accepts the target class objects while minimizing the probability of accepting outlier objects.

In literature, many machine learning applications like outlier detection, novelty detection, and anomaly detection have originated with the similar concept (Ritter and Gallegos, 1997; Bishop, 1993; Pauwels and Ambekar, 2011). Several algorithms have been developed based on SVM and ELM to address OCC problems (Tax and Duin, 2004; Schölkopf et al., 1999; Gautam and Tiwari, 2016).

In binary/multiclass classification, a model is trained to classify data objects into two or more classes, where each object is assigned to only one-class label. The trained model then classifies new data by defining a decision boundary around each class, such that it only accepts the associated class objects, while minimizing the probability of accepting other class objects. In a classification problem, the associated (optimal) class membership of a data object is predicted based on the probabilistic distribution of its association with each class.

5.2: Supervised vs. unsupervised learning

Supervised machine learning trains a model with an input X which represents a set of data objects and a labeled output Y which represents a categorical or continuous target value to construct a hypothesis function which can be later used to predict the output value for new data objects. Supervised learning algorithms address both classification and regression problems in machine learning. It is different from unsupervised learning, as the predictive model learns from the input as well as its corresponding output value.

In unsupervised learning, a model constructs a hypothesis based on input data X with no labeled output. Clustering is a conventional unsupervised machine learning algorithm, which groups observations to identify hidden patterns in the input data.

• • The raw datasets are obtained from the Public Health Agency of Canada (2020).
• • Weka software is used to analyze the dataset and run machine learning algorithms (Frank et al., 2016).
• • Excel software is used to visualize the data in two-dimensional (2D) format.

6.2: Case study 1: Diabetes data analysis

Diabetes is a disease that occurs when blood glucose reaches a very high level. The food we consume gets converted into blood glucose, which is our primary source of energy. Insulin, which is a hormone produced by the pancreas, helps glucose from food get into our cells to be utilized as energy. Sometimes our bodies do not make enough insulin or use the insulin well. The glucose stays in our blood, and cannot reach our cells (Health Information, 2016). According to the Public Health Agency of Canada, one in seven Canadians are affected by disease, and in 2050 about one in three will be afflicted (Taylor, 2016). The condition is progressing very rapidly, resulting in overflow in hospital visits. Hospitals are experiencing capacity, and resources issues that are profoundly affecting performance. Aside from hospital visits, there would be an overflow in clinic visits to the general and specialist physicians. It would be beneficial to have a reliable method for predicting how many diabetic patients are expected to be hospitalized. In a more general term, it would be very helpful to predict the future trends in the healthcare industry. Data mining is a unique method of data analysis that allows analysts to uncover hidden patterns in datasets. There are several software programs that are currently being used to conduct data analytic tasks, such as Weka, IBM Watson Analytics, and Alteryx. In this section, we used Weka to conduct our analysis. Weka is a collection of machine learning algorithms for data mining tasks. It can either be applied directly to a dataset or called from a separate Java code. Weka features include machine learning, data mining, preprocessing, classification, regression, clustering, association rules, experiments, and more. Weka is written in Java, and was developed at the University of Waikato in New Zealand.

In our case study, we downloaded raw datasets from the Public Health Agency of Canada. Although the data are huge and contain information on many prominent chronic diseases, our main focus is diabetes. Our choice to focus on diabetes in this case study was based on how prominent the disease is in today’s society and the significant growth rate associated with it. Our analysis targets predicting the number of diabetic patients who are expected to visit a specific hospital 1 year ahead. The dataset contained information on visits to hospitals for 2000–11, throughout the country. The list of features includes the patients’ age group, number of incident cases per year, number of GP visits per year, number of prevalent cases per year, and mortality per year. To create a generalized model that would be accepted and useful to healthcare administrators, we created six different models for 2006–11, which would effectively model the changes and fluctuations that were established throughout the years. In the data processing step, we converted all data types into strictly numerical values using the NominaltoNumeric filter package that is supported by the Weka software. After analyzing that dataset correlation, we decided to use seven different algorithms to predict the number of diabetic patients who are expected to visit a specific hospital 1 year ahead. These algorithms are: linear regression, decision trees, random forest, Naïve Bayes, support vector machine (SVM), and sequential minimal optimization (SMO).

• • Linear regression is a machine learning algorithm based on supervised learning. It performs a regression task. Regression models a target prediction value based on independent variables. It is mostly used for finding out the relationship between variables and forecasting (Gupta, 2018). Different regression models differ based on the kind of relationship between dependent and independent variables under consideration and the number of independent variables being used.
• • Decision tree algorithm belongs to the family of supervised learning algorithms (Lior and Oded, 2005). Unlike other supervised learning algorithms, the decision tree algorithm can be used for solving regression and classification problems too. The goal of using a decision tree is to create a training model that can be used to predict the class or value of the target variable by learning simple decision rules inferred from the training data.
• • Random forest as its name implies, random forest consists of a large number of individual decision trees that operate as an ensemble. Each individual tree in the random forest spits out a class prediction and the class with the most votes becomes our models prediction.
• • Naïve Bayes is a classification technique based on Bayes theorem with an assumption of independence among predictors (Ray, 2020). Naïve Bayes model is easy to build and particularly useful for very large datasets. Along with simplicity, Naïve Bayes is known to, sometimes, outperform other highly sophisticated classification methods.
• • SVM is a supervised machine learning algorithm, which can be used for both classification or regression challenges. However, it is mostly used in classification problems. In the SVM algorithm, we plot each data item as a point in n-dimensional space (where n is number of features we have) with the value of each feature being the value of a particular coordinate. Then, we perform classification by finding the hyperplane that best differentiates the two classes.
• • SMO is a method of decomposition, by which an optimization problem of multiple variables is decomposed into a series of subproblems each optimizing an objective function of a small number of variables, typically only one, while all other variables are treated as constants that remain unchanged in the subproblem.

Table 1 shows the performance evaluation of different prediction models trained with all features using different machine learning algorithms available within the Weka software. SMO has the best performing results, in terms of accuracy as well as execution time. The great advantage of the SMO approach is that we do not need a quadratic problem solver to solve the problem and instead it can be solved analytically. As a consequence, it does not need to store a huge matrix, which can cause problems with machine memory. Moreover, SMO uses several heuristics to speed up the computation, which is evident in the execution time. The SMO algorithm is, therefore, used throughout this project.

Fig. 2 shows an example model produced by the SMO algorithm to predict the number of diabetic patients who are expected to visit the hospital in the year 2006 based on records of the previous 5 years.

To evaluate the performance of a regression algorithm, we consider the following factors:

• • Root mean square error (RMSE) is one of the standard ways to measure the error of a model in predicting quantitative data.

  $\begin{array}{l}\hfill RMSE=\sqrt{\left.\frac{1}{n}\right)\sum _{i=1}^n{(y_i-x_i)}^2}\end{array}$

    (6)

  where n represents the number of features in that particular subset. (y_i − x_i) represent the differences between the experimental result and the actual value and then squared. The summation of all the values of the training set are expressed by sigma.
• • Mean absolute error (MAE) measures the average magnitude of the errors in a set of predictions, without considering their direction.

  $\begin{array}{l}\hfill MAE=\left.\frac{1}{n}\right)\sum _{i=1}^n\left.y_i-x_i\right|\end{array}$

    (7)

  where n also represents the number of features in that particular subset that you are working with. y_i − x_i also represents the difference between the experimental result and the actual value, but instead here it is absolute and cannot be negative.

Both MAE and RMSE express average model prediction error in units of the variable of interest. Both metrics can range from 0 to ∞ and are indifferent to the direction of errors. They are negatively oriented scores, which mean the lower the values the better the results.

Although Weka software provided the RMSE and MAE values for each model, it is still crucial to validate the models statistically, which was done in Microsoft Excel. We calculated the difference between the actual value of a given year and the predicted value for the same year. We were able to calculate the absolute error, average accuracy, and residuals. Using these values, we can manually calculate RMSE and MAE. We were also able to create a graphical representation of the error and accuracy. Fig. 3 shows a graphical representation of the accuracy (top) and the error (bottom) for the 2011 model when calculated manually.

In this case study, we created an individual model to predict the number of diabetic patients who are expected to visit the hospital in a specific year based on the previous 5-year records. Since we have records from 2006 to 2011, we were able to create six different models, one to predict each year. Our goal was to create a more generalized model that can be used for any year based on the previous 5-year records. Create such a generalized model that can maintain a low error rate was a challenging task. The normalized coefficients that were multiplied by the features were close in range, therefore, the method of choice was to average these coefficients for all the years and place the new averaged value as the new coefficient in the generalized model.

Once the generalized model is created, a cross-validation process is done to calculate the RMSE, MAE, error rates, and accuracy of this new model. We tested the new model and compared it with the individual models. Fig. 4 shows the accuracy (top) and the error (bottom) for the 2011 model when using the generalized model.

These average accuracy using the generalized model throughout the separate years is 93% compared to an average accuracy of 96% using each individualized model. When working with these types of scenarios, we expect the accuracy to drop much further when creating a general model. However, in our case, the model still shows an acceptable performance and could be used in later years.