Task (T)

The task refers to what we want the machine learning model to do. The function the model learns is the task. This does not refer to the task of actually learning but rather the task at hand. For example, a robotic hoover would have the task of hoovering a surface. Often a task is broken down into smaller tasks.

Performance (P)

Performance is a quantitative measure of a machine learning model’s ability. Performance is measured using a cost function, which typically calculates the accuracy of the model on the task.

Performance can be measured by an error rate, which is the proportion of examples for which the model gives the wrong outputs. The learning method aims to minimize the error rate, ideally without falling into local minima or maxima.

Experience (E)

Experience refers to the amount of labeled data that is available as well as the amount of supervision required by the machine learning model.

Mitchell’s reference proves useful in defining machine learning projects. This definition helps understand the data required for collection (E), which develops as the knowledge base; the problem at hand that needs a decision (T); and how to evaluate the output (P).

For example, a prediction algorithm may be assigned task T to predict peak emergency hospital admissions based on experience data E of past hospital admissions and their respective dates and times. The performance measure P could be the accuracy of prediction; and as the model receives more experience data, it would ideally become better in its forecast. It is noteworthy that one criterion does not necessarily determine performance.

Machine learning is used to answer questions such as “Is it likely I have type 2 diabetes?”, “What object is this in this image?”, “Can I avoid the traffic?”, and “Will this recommendation be right for me?”

Machine learning is employed favorably in each of these problem domains, yet the complexity of real-world problems means that it is currently infeasible for a specialized algorithm to work faultlessly every time, in every setting, for everyone—whether in healthcare or any other industry.

A common aphorism in math comes from British statistician George Box: “All models are wrong, but some are useful.” The purpose of machine learning is not to make perfect guesses, as there is no such thing as an ideal model. Instead, machine learning builds on foundations in statistics to provide useful, accurate predictions that can be generalized to the real world.

When training a machine learning algorithm, the training dataset must be a statistically significant random sample. If this is not the case, there is a risk of discovering patterns that don’t exist, or the noise. The memorization of training data is known as overfitting. The model remembers the training data, performing well in training but not with previously unknown data. Equally, if the training data is too small, the model can make inaccurate predictions.

The goal of machine learning, whether supervised or unsupervised, is generalization—the ability to perform well based on past experiences.

Selecting the Right Features

There are two different main approaches when it comes to feature selection. First is an independent assessment based on the general characteristics of the data. Methods belonging to this approach are called filter methods, as the feature set is filtered out before model construction.

The second approach is to use a machine learning algorithm to evaluate different subsets of features and finally select the one with the best performance on classification accuracy. The latter algorithm will be used in the end to build a predictive model. Methods in this category are called wrapper methods because the arising algorithm wraps the entire feature selection process.

The law of large numbers theorem applies to machine learning prediction. The theorem describes how the result of a large number of identical experiments will eventually converge or even out.

This process is iterated until the system converges on the best values for a and b. As such, the machine learns through experience and is ready for deployment. Typically, we are concerned with minimizing the error of a model or making the most accurate predictions possible.

Within supervised machine learning, there are two main approaches: regression-based systems and classification-based systems.

Classification

Classification is the process of determining a category for an item (or items, x) from a set of discrete predefined categories (or labels, y) through approximating a mapping function, f. Classification algorithms use features (or attributes) to determine how to classify an item into at least a binary class or two or more classes.

It is from the data that classification algorithms learn to classify. Classification problems require examples to be labeled to determine patterns; hence, a model’s classification accuracy is best validated through the correctly classified examples of all predictions made.

Algorithms that perform classification include decision trees, naïve Bayesian, logistic regression, kNNs (k-nearest neighbors), support vector machine, and so forth.

Regression

Regression involves predicting an output that is a continuous variable. As a regression-based system predicts a value, performance is measured by assessing the number of prediction errors.

The coefficient of determination, mean absolute error, relative absolute error, and root mean square error are statistical methods used for evaluating regression models.

Regression algorithms include linear regression, regression trees, support vector machines, k-nearest neighbor, and perceptrons.

It is often possible to convert a problem between classification and regression. For example, patient HbA1c can also be useful if stored as a continuous value. A continuous value of HbA1c = 6.9% can be classified into a discrete category, diabetic, on rules defined on HbA1c as Diabetic (Set 1), HbA1c ≥ 6.5%, and Non-diabetic (Set 2), HbA1c < 6.5%.

Time series refers to input data that is supplied in the order of time. The sequential ordering of the data adds a dimension of information. Time series regression problems are commonplace.

Iterative Dichotomizer 3 (ID3)

Python, scikit-learn; method, decision-tree-id3

Ross Quinlan has contributed significantly to the area of decision tree learning, developing both ID3 and C4.5 [76]. The ID3 algorithm uses the cost function of information gain, based on work from “the father of information theory” Claude Shannon. Information gain is calculated through a measure known as entropy.

Entropy determines how impure, or disordered, a dataset is. Given an arbitrary categorization C, into categories {c1, … cn}, and a set of examples S, for which the proportion of ci is pi, the entropy of sample S is the following:

Information gain is calculated using entropy. This value is given for attribute A, with respect to examples S. The values attribute A can take {t1, … tn}, representing the totality of set T.

ID3 selects the node with the highest information gain to produce subsets of data that are then applied the ID3 algorithm on recursively. The information gain of an attribute can be understood as the expected loss in entropy as the result of learning the value of attribute A. The algorithm terminates once it exhausts the attributes or the decision tree classifies the examples completely. ID3 is a classification algorithm that is unable to cater for missing values.

C4.5

Python, scikit-learn; method, C45algorithm

C4.5 is an enhancement built on ID3 that takes number and size of branches into account when choosing an attribute. C4.5 can handle continuous and discrete labels and uses gain ratio as its splitting criteria. It improves on the execution time and accuracy as the size of the dataset increases.

C4.5 introduces pruning—a bottom-up technique also known as subtree raising—and subtree replacement to ensure the model doesn’t overfit the data.

CART

Python, scikit-learn; method, cart

CART, which stands for Classification and Regression Trees, is a decision tree algorithm which uses the Gini index as the cost function. Represented as a binary tree, as with C4.5, CART can be used for regression and classification problems. While entropy is used for exploratory analysis, Gini is to minimize misclassification. Root nodes represent input variable (x), and leaf nodes contain the output variable (y) used to make a prediction.

Bagging

Bagging is an ensemble technique that involves creating multiple independent models with datasets using the bootstrap sampling technique (see Figure 4-2). The technique aids in reducing error by reducing variance. In general, bagging improves predictions as anomalous behaviors are averaged out. In comparison, a collection of iterative predictive models is known as boosting.

In bagging, n′ < n samples are taken from dataset D with replacement. A random selection of features for constructing the best split is chosen each time.

An average, majority vote, or statistical method, is used to determine the average prediction from a model. Random forest decision trees (see Figure 4-3) extend the concept of bagging, as they use a random subset of features too, resulting in less correlation among predictions from learners.

Boosting

Boosting is an ensemble method that produces a collection of predictive models iteratively. The concept involves each new model learning from the errors of previous models.

Boosted methods are typically implemented when there is a plethora of data for prediction.

Through iteratively creating new models to compensate for error, the process “boosts” or converts weak learners into strong learners—reducing their bias and variance. Each iteration of the technique learns relationships in the data and is then subsequently analyzed for errors, with incorrect classifications given a weighting. The aim is to minimize the error from the previous model. The underlying machine learning method used for boosting algorithms can be anything.

Perceptron

Perceptrons are the fundamental units of ANNs. In AI, the perceptron is a node or unit that takes multiple inputs and has a single output, typically 1 or –1. Perceptrons take a weighted sum of the inputs, S, and a unit function calculates the output for the node (see Figure 4-9). Unit functions can be the following:

• Linear—such as the weighted sum

• Threshold functions that only fire if the weighted sum is over a threshold

• Step functions that output the inverse, typically –1, if S is less than the threshold

• Sigma function (1/(1 + e – s), which allows for backpropagation

Perceptrons mimic biological neurons (see Figure 4-10), where dendrites receive an input. If the signal is high enough within a set duration of time, the neuron sends an electrical pulse to the terminals then received by dendrites of other neurons. Each perceptron has a bias, similar to b of the linear function y = ax + b. It moves the line up and down to fit the prediction with the data better.

where 1 is x₀ and b = w₀.

Artificial Neural Networks

Python, scikit-learn; method, linear_model.Perceptron

Artificial neural networks are made of perceptrons and contain one or more hidden layers (see Figure 4-11). There are several topologies, with the simplest being a feedforward network.

Backpropagation is the method of determining the error, or loss, at the output and propagating it back to the network. Weights at each node are updated to minimize the respective error output from each neuron. Backpropagation seeks to minimize the error for neurons and thereby the entire model through updating respective neuron weights.

The primary learning that must be accomplished in neural network models is training neurons when to fire. An epoch refers to one training iteration of forward- and backpropagation. During the learning phase, nodes in a neural network adjust their weights based on the error of the last test result. The learning rate controls how fast the network learns.

The number of perceptrons or neurons in a model are equal to the number of variables in the data. Some representations may also have a bias node.

Typically, artificial neural networks have neurons organized in layers. Each layer can perform different transformations on the input data it receives.

Feedforward Neural Network

In a feedforward neural network, the sum of the weighted inputs is fed to the output (see Figure 4-13). The output is activated (typically 1) when the sum exceeds the unit function threshold. If it doesn’t fire, it typically outputs –1.

Recurrent Neural Network (RNN): Long Short-Term Memory

Recurrent neural networks or RNNs save the output of a layer and feed it back to the input to improve predictions of the output layer (see Figure 4-14). As such, each node has a memory when performing computations and makes use of sequential information. As a result, nodes are deemed to have a memory.

Convolutional Neural Network

Convolutional neural networks are deep, feedforward neural networks. Layers are known as convolutional layers and are often used in speech recognition, spatial data, NLP, and computer vision problems. Separate convolutional layers typically apply to different aspects of a particular problem.

For instance, in understanding whether there is a face present within a photograph, separate convolutional layers within a DNN may be used to identify different aspects of the face—the eyes, nose, ears, mouth, and so forth.

Modular Neural Network

Similar to the human brain, the concept of modular neural networks is that of neural networks working together.

Radial Basis Neural Network

This is a neural network that uses radial basis activation functions where the value depends on the distance from the origin (see Figure 4-15).

The key strength of ANNs and deep learning lies in the ability to generate complex linear and nonlinear models from the training dataset.

DNNs (deep and multilayered ANNs) can learn complex relationships between data from training data alone, which typically improves generalization. DNNs can still succumb to the problem of overfitting data, so appropriate pruning may be required. As the name suggests, deep networks can also take a long time to train and require a large number of training examples.

A common concern with neural networks, particularly in healthcare, is that they act as a black box and do not present a clear interface for a user to understand findings. Practically, an ANN model is difficult to interpret, and weights and biases aren’t readily interpretable to what is important in the model.

It is typically considered that for supervised learning problems, the upper bound on neurons to prevent overfitting is

N_h=N_s(〈 ∗(N_i + N_o)),

Clustering

k-Means

Python, scikit-learn; method, cluster.KMeans

k-Means clustering (see Figure 4-16) aims to find k similar groups within the data. k-Means is an iterative algorithm that calculates the centroids of the defined k clusters, with all training data assigned to a cluster.

Data points are assigned by the distance between the data point and its centroid. A cluster centroid is a collection of attribute values that define the resulting clusters.

where c_i is the collection of centroids in set C and distance (dist) is standard Euclidean distance.

Rather than defining groups before data exploration, clustering enables the model to determine groups that have formed organically.

Association

Association rule learning methods extract rules that best explain perceived relationships between variables in data. These rules can discover useful associations in large multidimensional datasets that can be used to drive and optimize.

Association rule learning is historically best applied to online shopping checkout basket datasets gathered on users’ purchasing habits. For instance, when someone buys a pizza, they may also tend to purchase wine, just like how someone who purchases lettuce may tend to buy tomatoes, cucumbers, and onion too. Through analyzing transactional datasets, the probability of associations can be predicted. We know that certain items (whether that be food, clothes, or even disease) frequently occur together, and association rule learning seeks to understand these relationships.

In healthcare, in particular, associative symptoms can be understood to predict better and diagnose disease and adverse events. Determining potential adverse effects based on medication and associative patient comorbidity pathways could lead to improved care and treatment pathways.

There are three important metrics to be familiar with.

Apriori

Apriori is the most popular association rule mining algorithm. The Apriori algorithm is frequently used in pattern mining and in the identification of features and items that occur together. It is used extensively in basket analysis and is a powerful tool to find hidden feature patterns. It is usually applied to transactional databases to mine frequent relationships and generate association rules.

Apriori begins with a set of n items. The algorithm calculates all possible candidate itemsets, or frequent itemsets, which meet the specified threshold support and confidence values. The Apriori property states for an itemset to be frequent, all of its subsets must also be frequent. The Apriori algorithm uses a bottom-up approach; frequent subsets are identified and increased one item at a time.

The Apriori method is relatively simple (see Figure 4-17):

• Apply minimum support to find all frequent sets of k items within dataset T where a frequent itemset has support ≥ min(sup).

• Expand selection to find frequent sets of k + 1 items using frequent k-itemset.

The maximum size of an itemset is k, the number of items.

Missing/Null Values

Missing data and null values are not a huge problem in isolation, but a growing number of empty values may help determine whether to drop a variable, ignore missing values, or compute a predicted value.

Most data scientists support the dropping of variables if an attribute has upward of 50% empty or null values. This threshold does vary.

Low Variance

Data attributes that are very similar to one another do not carry much information. As a result, there is little variance within the data. Hence, a lower variance threshold can be set and dimensions removed based upon this.

As variance is dependent on range, data normalization should take place first.

High Correlation

Attributes with similar data trends are likely to carry similar information. Multicollinearity is the carrying of similar information that can reduce the performance of the model. Correlation between continuous data is determined by the Pearson product-moment coefficient, which is a measure of the linear correlation between two variables X and Y.

For discrete data, the Pearson’s chi-square value determines how likely it is that any observed difference between sets arose by chance.

Random Forest Decision Trees

Random forest decision tree ensembles are useful for identifying key features. Random forest decision trees use a subset of features that can identify best split attributes.

If an attribute is often selected as the best split node, it is likely to be a feature to keep. Decision trees are particularly useful for visualizing reductions also.

Backward Feature Elimination

In Backward Feature Elimination , the model trains on n attributes. Starting with n attributes, at each iteration, the model is trained on n-1 attributes n number of times. The attribute that demonstrates the smallest increase in error rate is removed, and the algorithm performs another iteration on n-1 attributes.

At each iteration k, a model with n-k attributes is trained. This is computationally expensive.

Forward Feature Construction

Forward Feature Construction is the opposite of Backward Feature Elimination. The model starts with one attribute and evaluates which of the attributes has the highest increase in performance. Much like it’s opposite Backward Feature Elimination, this method is computationally expensive unless operating in a lower range of dimensions.

Principal Component Analysis (PCA)

Principal component analysis decreases the number of variables (axes) through transforming an original set of variables into a new set, or the principal components (see Figure 4-19). This process is iteratively calculating the maximum variance within features in the data.

Each component is orthogonal and a linear combination of original variables. Orthogonality indicates that the correlation between the components is zero.

The first principal component is the linear combination of the original dimension that has the maximum variance; and the nth principal component is the linear combination with the highest variance, subject to being orthogonal to the n – 1 principal components .

Getting Started with NLP

Giving an NLP model an input sentence and receiving a useful output requires several key components (see Figure 4-20).

Noise Removal

Noise removal involves preparing a dictionary of noisy tokens (i.e., words) and parsing the text, removing the tokens found in the noise dictionary. For instance, words like the, a, of, this, that, and so forth would be removed.

Lexicon Normalization

Follow, following, followed, and follower are all variations of the word follow. Contextually, the words are similar. Lexicon normalization reduces dimensionality through stemming, which strips suffixes and prefixes, and lemmatization, which is a defined procedure that uses word structure and grammar relationships.

Porter Stemmer

Python, NLTK; method, PorterStemmer

The Porter stemmer algorithm is a popular and useful method to improve the effectiveness of information retrieval [81]. The algorithm works on the principle that many words in English share a common root. Stemming works on suffixes and removes common morphological and inflexional endings from words. Therefore, stemming allows one to reduce similar words into a common root form.

For example, take the text “I felt troubled by the fact that my best friend was in trouble. Not only that, but the issues I had dealt with yesterday were still troubling me.”

The words troubled, trouble, and troubling all share the common root trouble. Therefore, according to the Porter stemmer algorithm, instead of counting all three words once, the stem trouble is counted thrice instead. The benefit of stemming is that common words can be clustered under a common stem to provide a more accurate statistical representation of the number of occurrences of a certain word. However, a drawback of stemming is that the semantic meaning of the word may be lost.

Stemming and lemmatization are used to reduce inflectional forms and sometimes derivationally related forms of a word to a common base form.

Object Standardization

A corpus of text may contain words that cannot be found in lexicon dictionaries. For example, on Twitter, someone may mention DM’ing someone, or another may like an RT of someone else’s tweet. Acronyms, hashtags, slang, and colloquialisms can be removed through prepared dictionaries or using regular expressions.

Dependency Parsing

Toolkit, NLTK; method, StanfordDependencyParser

Sentences are composed of words in a structure. Basic grammar can determine the relationship or dependencies between a structure. Dependency parsing represents this in a tree structure and represents grammar and arrangement of words (see Figure 4-21).

Dependency grammar analyzes asymmetrical binary relationships between tokens. The Stanford parser from NLTK is commonly used for this purpose.

Using the example in Figure 4-21, the parse tree determines the root of the word as “saw” and is then linked by subtrees. The subtrees are split by subject and object, with each subtree also showing dependencies .

Part of Speech Tagging

Toolkit, NLTK; method, word_tokenize

Part of speech (POS) tagging involves associating each word or token in a sentence with a part of speech tag. These tags are the basic English labels that you learned in primary school and determine nouns, verbs, adjectives, adverbs, numbers, and so on.

This is particularly useful for tasks such as building parse trees—which can, in turn, be used for determining what something is, sentiment analysis, determining appropriate answers to questions, or understanding similar entities (see Figure 4-22).

Part of speech tagging is the basis of the assists in several areas .

N-Grams

Python: NLTK; Method: ngrams

N-Grams are used in many NLP problems. If X = number of words in a given sentence K, the number of n-grams for sentence K would be

N-Grams preserve the sequences of N items from the text input. N-Grams can have a different N value: unigrams for when N = 1, bigrams when N = 2, and trigrams when N = 3.

N-Grams are used extensively for spelling correction, word decomposition, and summating texts.

TF–IDF Vectors

A document can be represented as a high-dimensional vector in the space of words. Each entry in the vector corresponds to a different term within the documents and the number of its occurrences. The TF–IDF (Term Frequency–Inverse Document Frequency ) vector scheme assigns each term within the document a weight using the following formula:

Each term’s weight (W_td) is calculated by multiplying the frequency of the term (f_td) by the log of the total number of documents (D) divided by the number of documents the term occurs at least once in (ND_t). The ordering of the terms may not necessarily be maintained.

Weights for the terms gathered can then be used to determine documents with high frequencies of the specific term within them. A collection of TF–IDF vectors could be used to represent a user’s interest.

Latent Semantic Analysis

Where there is a corpus of a large number of documents, for each document d, the dimension of the vector representing each document can typically exceed several thousand. Latent semantic analysis relies on the fact that intuitively, terms in documents may often be related. For example, if document d contains the term sea, it will often contain the word beach.

Equivalently, if the vector representing d has a non-zero component in the entry for sea, it will also have a non-zero component in the beach entry. If this kind of structure can be detected, relationships between words can be automatically learned from the data.

The word document is represented by a matrix A, which is decomposed using singular value decomposition to give the strength of the most significant correlations and their directions.

The decomposition of A allows one to discover the semantics of the document through correlations between terms and their significance within the document. The latent semantic analysis method can be applied to determine the context of a variety of materials to present a web user with results that are contextually relevant.

Cosine Similarity

Python, SciPy; method, cosine

The cosine similarity is a metric that measures the similarity between two vectors. Therefore, this could either be used to define the similarity between two documents or a document and a query.

The cosine similarity between two vectors can be defined as the following:

The similarity between two vectors is calculated as the inner product of the vectors divided by the product of the lengths of the vectors in question. Intuitively, the greater the angle between two documents, the less similar they are. This holds for vectors in any N-dimensional space.

Furthermore, the TF–IDF vector scheme can be integrated with the cosine similarity metric to determine the similarity between a search query and a plethora of documents:

The similarity between a query q and document d is calculated as the product of the TF–IDF weights for the term in both the document and query summed over all the terms in both the query and document; this is divided by the product of the length of the document and length of the query. Practically, however, calculating Sim(q,d) would prove computationally expensive as the number of documents used grows.

Naïve Bayesian Classifier

The Bayesian classifier is based on the Bayesian theorem and is particularly suited when the dimensionality of the inputs is high. Despite its simplicity, naïve Bayes can often outperform more sophisticated classification methods.

Given a specified threshold, this method can be used to classify the probability of whether a vector representing a document is of interest to a user. Given an attribute d, we can calculate whether the example belongs in class C with the following formula:

Other techniques such as kNN and ANN can also be used to classify and retrieve information.

Good Data Management

With the volume and complexity of data used in machine learning tasks, proper data management is paramount. Not only does this involve having the processes, policies, procedures, permissions, and certifications required for data management, it consists of the management and transformation of training data to learn an optimal model.

Establish a Performance Baseline

It is crucial to have a baseline to measure the performance of your algorithm against other iterations and model types. As there is no one perfect algorithm for every problem, attempt many algorithms to identify the relative performance of your models.

Spend Time Cleaning Your Data

The time required to train a machine learning model varies significantly between algorithms. The accuracy of data and time spent on training the model positively influence model accuracy. Take time to clean your data to ensure predictions are as robust as possible.

It is often the case that not all of the input variables impact the dependent variable or outcome. Ensure the variables used by the model do not include those that are irrelevant.

Training Time

If the dimensionality of data is large and computing power limited, training time will be extensive. When time is short, it is useful to consider machine learning algorithms that do not require substantial training. For example, neural networks may not be appropriate for time-limited tasks.

Choosing an Appropriate Model

Some machine learning algorithms make particular assumptions about the structure of data or the desired outputs. For this reason, it is helpful to consider the model choice and whether it is an appropriate approach. Utilizing the right model has a plethora of benefits, including more accurate predictions, faster training times, and more useful results.

Some machine learning models are resistant to outliers or nonparametric tests. For example, decision tree–based approaches typically classify a node into two based on a threshold. Data outliers are therefore less impactful in tree-based approaches.

Evaluating a range of models is a useful approach to machine learning. Occam’s razor is typically applied to choosing the model of choice. Occam’s razor seeks to identify the most straightforward model that achieves the desired output given all else being equal.

Choosing Appropriate Variables

Although more data is ordinarily invited in machine learning problems, it is typically preferable to work with fewer predictor variables for many reasons.

Redundancy

Increasing the number of variables within a training dataset increases the chances of models learning hidden relationships between them. It is vital to identify unnecessary variables and only use nonredundant predictor variables within models. A model that is learning redundant connections will impact model accuracy.

Overfitting

Even if there are no relationships among predictor variables within a model, it is still beneficial to use fewer variables. Complex models, or those that use a high number of predictor variables, typically suffer from overfitting.

Models perform well on training datasets but are less accurate on validation and real-world settings, as they learn the error within the data (i.e., the noise) rather than the signal, or relationships, between variables.

Productivity

Even if all of the variables within a sophisticated machine learning model are relevant, there is a practical impact in using a large number of predictor variables that can affect productivity.

Practical considerations include the amount of data available, the subsequent effect on storage, computing resources, associated costs, time allocated for the project, and the time required for learning and validation. Predictor variables can be identified through feature selection, or transformed through feature extraction. SVMs are useful in cases of increased data dimensionality.

The Pareto principle (or the 80/20 rule or law of a vital few) is a useful measure to use in machine learning projects. Using the Pareto principle, focusing on the 20% most significant predictor variables should facilitate the building of relatively successful models within a reasonable time. Microsoft learned that 80% of the errors and crashes in Windows and Office were caused by 20% of the entire set of bugs detected [82].

Understandability

Models with fewer predictor variables are easier to visualize, understand, and explain. A key aspect of a successful machine learning project is that all stakeholders can comprehend the model. This often requires data scientists trading off.

By reducing the number of predictor variables, there is a chance there may be a reduction in the success of a machine learning model. However, this concurrently makes the model easier to interpret and understand.

The usefulness of this approach is only realized toward the end of a project when it comes to sharing not only the performance of a model but also how it works. This is particularly relevant in healthcare, as there are concerns about using black box models.

Accuracy

Any machine learning model aims to generalize well. Depending on the use case, an approximation may be more beneficial than a precise, accurate output. Through approximating values, models tend to avoid overfitting and reduce the amount of time spent on processing.

Impact of False Negatives

When evaluating the impact of a model before deployment into a live environment, consider the impact of false negatives. For example, take a predictive model that classifies the risk of breast cancer. A false positive would mean that a patient is informed they have breast cancer when they do not, which should be identified later in the treatment pathway. However, a false negative would mean a patient with breast cancer would not be notified—which is potentially far worse and costly.

Linearity

Many machine learning algorithms assume that relationships are linear—in other words, that classes can be separated by a straight line of best fit or its higher-dimensional representation. There are instances where relationships are not linear. Relying on a linear classification algorithm where there is a nonlinear class boundary will result in low accuracy.

Linear regression, for example, assumes both input and output variables do not contain noise. It is vital to expose the signal within data to ensure models learn the signal or the correct relationships within the data. It is advantageous to identify and remove outliers for the output variable in particular to reduce the chances of learning the noise.

Data with a nonlinear trend may require transforming to make the relationship linear, for instance, log-transforming data where there is an exponential relationship.

Linear algorithms are typically the first methods attempted in machine learning scenarios. As a result of linearity, algorithms are simple and quick to train.

Parameters

Each machine learning model is subject to parameters and hyperparameters. The time and resource required to train a model increases as the number of parameters does. An algorithm’s parameter is a variable that is utilized by the model where the value can be estimated from the dataset.

An algorithm’s hyperparameter is a variable that is external to the model and used to estimate parameters. A hyperparameter’s value cannot be estimated from data. Instead, it is set by the data scientist, randomly or otherwise, or through the use of heuristics.

Ensembles

In specific machine learning problems, it may prove more useful to group classifiers together using the techniques of voting, weighting, and combination to identify the most accurate classifier possible. Ensemble learners are very useful in this aspect.

Predicting Blood Glucose

Georga, E. I. et al. used random forest decision trees on many features collected from 15 people with type 1 diabetes to predict short-term subcutaneous glucose concentrations [85]. Glucose concentration was predicted through the use of SVMs.

The study concluded that the inclusion of two biomarker features—8-hydroxy-2-deoxyguanosine, an oxidative stress marker, and interleukin-6—improved classification accuracy.

Predicting Risk

Farideh Bagherzadeh-Khiabani et al. used the clinical dataset from 803 female prediabetics consisting of 55 features collected over a decade to develop multiple models for predicting type 2 diabetes risk [86]. The study developed a logistic model, demonstrating wrapper models, or models that consider a subset of features and improve the performance of clinical prediction models. This was developed into an R program to visualize output.

Summers, Panesar, et al. developed a model consisting of 34 features collected for people with prediabetes and obesity to predict risk of developing type 2 diabetes based on the health and engagement outcomes of over 200,000 people who used the Low Carb Program digital health intervention. This was developed in Python and deployed on the cloud to assess patient risk and suitability for the award-winning digital therapy [16, 58].

Razavian et al. conducted another extensive study, determining that type 2 diabetes prediction models could be accurately derived from population-scale datasets [87]. Over 42,000 variables were collected from over 4 million patients between 2005 and 2009. Machine learning was used to determine relevant features, which tailed out at around 900. Novel risk factors for type 2 diabetes were identified, such as chronic liver disease, high alanine aminotransferase, esophageal reflux, and history of acute bronchitis.

Duygu Çalişir et al. developed an automatic diagnosis system for type 2 diabetes with a classification accuracy of almost 90% through the use of linear discriminant analysis and SVM classifiers [86]. Latent Dirichlet Allocation (LDA) was used to distinguish features between healthy patients and patients with type 2 diabetes, which was then in turn used as the input for the SVM classifier. A tertiary stage evaluated sensitivity, classification, and confusion before determining the output.

Predicting Risk of Other Diseases

Machine learning has likewise been applied to diagnosing the risk of other diseases related to diabetes.

Lagani et al. identified the least set of clinical variables with the best predictive accuracy for complications such as cardiovascular disease (heart disease or stroke), hypoglycemia, ketoacidosis, proteinuria, neuropathy, and retinopathy [88].

Huang et al. used decision tree–based prediction models on identifying diabetic nephropathy in patients [89]. Leung et al. compared several methods: partial least square regression, regression trees, random forest decision trees, naïve Bayes, neural networks, and SVMs on genetic and clinical features [90]. Patient age, years of diagnosis, blood pressure, genetic polymorphisms of uteroglobin, and lipid metabolism arose as the most efficient predictors of type 2 diabetes.

Summers et al. developed a predictive model utilizing a dataset consisting of over 120 features to identify comorbid depression and diabetes-related distress. Summers et al. explored several techniques including random forest decision trees, SVMs, naïve Bayes, and neural networks. Patient age and employment status were defined as the variables most likely to affect predictive accuracy [16, 86].

Unsupervised learning has mainly concentrated on association rule mining, identifying associations between risk factors in people with diabetes. Simon et al. proposed an extension to association rule mining labeled survival association rule mining, which included survival outcomes, and made adjustments for confounding variables and dosage effects [86].

Reversing Disease

The understanding of type 2 diabetes has been radically disrupted by the generation of real-world evidence over the last 15 years. Previously, type 2 diabetes was understood as a chronic, progressive disease.

The Low Carb Program digital health intervention has been instrumental in demonstrating that sustainable weight loss is achievable through scalable digitally delivered therapies, supporting prediabetes and type 2 diabetes remission. The platform’s peer-reviewed evidence shows 54% reduce or eliminate diabetes medications, with retention of 71% at 1 year.

A key driver in the Low Carb Program’s success has been the implementation of a predictive model to identify education modules, resources, recipes, and discussion threads that are most likely to engage patients. This results in the app itself showing a different experience to each patient based on key features including ethnicity, age, gender, location, language, weekly food budget, dietary preferences, health status, and comorbidities.

This uses the patient’s data to provide personalized, tailored care to best empower decisions and provide around-the-clock support. The benefits of this approach are self-evident, with one in four patients achieving type 2 diabetes remission at 1 year. Outcomes from Low Carb Program have been used to influence public nutrition policy and guidelines. The app was ranked the #1 app for diabetes in the NHS [91, 92].

Machine learning is so often focused on predicting and diagnosing disease. The application of machine learning to reverse disease provides hope that we can support improved behaviors and redefine outdated paradigms.