Statistics at the Start

Machine-learning methods have changed rapidly in the past several years, but a larger trend began about a decade ago. Specifically, the field of data science emerged and we experienced an evolution from statisticians to computer engineers and algorithms (see Figure 10-1).

Figure 10-1. The evolution from statisticians to computer engineers and algorithms

Classical statistics was the domain of mathematics and normal distributions. Modern data science is infinitely flexible on the method or properties, as long as it uncovers a predictable outcome. The classical approach involved a unique way to solve a problem. But new approaches vary drastically, with multiple solution paths.

To set context, let’s review a standard analytics and split a dataset into two parts, one for building the model, and one for testing it, aiming for a model without overfitting the data. Overfitting can occur when assumptions from the build set do not apply in general.

For example, as a paint company seeking homeowners that might be getting ready to repaint their houses, the test set may indicate the following:

Understandably, you cannot generalize on this property. But you could look at income pattern, data regarding the house purchase, and recently filed renovation permits to create a far more generalizable model. This kernel of an approach spawned the transition of an industry from statistics to machine learning.

The “Sample Data” Explosion

Just one generation ago, data was extremely expensive. There could be cases in which 100 data points was the basis of a statistical model. Today, at web-scale properties like Facebook and Google, there are hundreds of millions to billions of records captured daily.

At the same time, compute resources continue to increase in power and decline in cost. Coupled with the advent of distributed computing and cloud deployments, the resources supporting a computer driven approach became plentiful.

The statisticians will say that new approaches are not perfect, but for that matter, statistics are not, either. But what sets machine learning apart is the ability to invest and discover algorithms to cluster observations, and to do so iteratively.

An Iterative Machine Process

Where machine learning stepped ahead of the statistics pack was this ability to generate iterative tests. Examples include Random Forest, an approach that uses rules to create an ensemble of decision trees and test various branches. Random Forest is one way to reduce overfitting to the training set that is common with simpler decision tree methods.

Modern algorithms in general use more sophisticated techniques than Ordinary Least Squares (OLS) regression models.

Keep in mind that regression has a mathematical solution. You can put it into a matrix and compute the result. This is often referred to as a closed-form approach.

The matrix algebra is typically (X’X)^–1X’Y, which leads to a declarative set of steps to derive a fixed result. Here it is in more simple terms:

If X + 4 = 7, what is X?

You can solve this type of problem in a prescribed step and you do not need to try over and over again. At the same time, for far more complex data patterns, you can begin to see how an iterative approach can benefit.

Digging into Deep Learning

Deep learning takes machine learning one step further by applying the idea of neural networks. Here we are also experiencing an iterative game, but one that takes calculations and combinations as far as they can go.

The progression from machine learning to deep learning centers on two axes:

• Far more complex transfer functions, and many of them, happening at the same time. For example, take the sng(x) to the 10th power, compare the result, and then recalibrate.

• Functions in combinations and in layers. As you seek parameters that get you closest to the desired result, you can nest functions. The ability to introduce complexity is enormous. But life and data about life is inherently complex, and the more you can model, the better chance you have to drive positive results.

For example, a distribution for a certain disease might be frequency at a very young or very old age, as depicted in Figure 10-2.

Classical statistics struggled with this type of problem-solving because the root of statistical science was based heavily in normal distributions such as the example shown in Figure 10-3.

Figure 10-2. Sample distribution of disease prevalent at young and old age

Figure 10-3. Sample normal distribution

Iterative machine learning models do far better at solving for a variety of distributions as well as handling the volume of data and the available computing capacity.

For years, machine learning methods were not possible due to excessive computing costs. This was exacerbated by the fact that analytics is an iterative exercise in and of itself, and the time and computing resources to pursue machine learning made it unreasonable, and closed form approaches reigned.

The Move to Artificial Intelligence

The move from machine learning to broader artificial intelligence will happen. We are already seeing the accessibility with open source machine learning libraries and widespread sharing of models.

But although computers are able to tokenize sentences, semantic meaning is not quite there. Alexa, Amazon’s popular voice assistant, is looking up keywords to help you find what you seek. It does not grasp the meaning, but the machine can easily recognize directional keywords like weather, news, or music to help you.

Today, the results in Google are largely based on keywords. It is not as if the Google search engine understands exactly what we were trying to do, but it gets better all the time.

So, no Turing test yet—we speak of the well-regarded criteria to indicate that a human cannot differentiate from a human or a computer when posing a set of questions.

Therefore, complex problems are still not likely solvable in the near future, as common sense and human intuition are difficult to replicate. But our analytics and systems are continuously improving opening several opportunities.