Can we really make our machines (that is, our computers) learn? In this and the next chapter, we’ll show exactly how that magic happens. What’s the “secret sauce” of this new application-development style? It’s data and lots of it. Rather than programming expertise into our applications, we program them to learn from data.

We’ll present many Python-based code examples that build working machine-learning models then use them to make remarkably accurate predictions. The chapter is loaded with exercises and projects that will give you the opportunity to broaden and deepen your machine-learning expertise.

Wouldn’t it be fantastic if you could improve weather forecasting to save lives, minimize injuries and property damage? What if we could improve cancer diagnoses and treatment regimens to save lives, or improve business forecasts to maximize profits and secure people’s jobs? What about detecting fraudulent credit-card purchases and insurance claims? How about predicting customer “churn,” what prices houses are likely to sell for, ticket sales of new movies, and anticipated revenue of new products and services? How about predicting the best strategies for coaches and players to use to win more games and championships? All of these kinds of predictions are happening today with machine learning.

Here’s a table of some popular machine-learning applications:

It’s difficult to know in advance which model(s) will perform best on your data, so you typically try many models and pick the one that performs best. As you’ll see, scikit-learn makes this convenient for you. A popular approach is to run many models and pick the best one(s). How do we evaluate which model performed best?

You’ll want to experiment with lots of different models on different kinds of datasets. You’ll rarely get to know the details of the complex mathematical algorithms in the sklearn estimators, but with experience, you’ll become familiar with which algorithms may be best for particular types of datasets and problems. Even with that experience, it’s unlikely that you’ll be able to intuit the best model for each new dataset. So scikit-learn makes it easy for you to “try ’em all.” It takes at most a few lines of code for you to create and use each model. The models report their performance so you can compare the results and pick the model(s) with the best performance.

Supervised machine learning falls into two categories—classification and regression. You train machine-learning models on datasets that consist of rows and columns. Each row represents a data sample. Each column represents a feature of that sample. In supervised machine learning, each sample has an associated label called a target (like “dog” or “cat”). This is the value you’re trying to predict for new data that you present to your models.

You’ll work with some “toy” datasets, each with a small number of samples with a limited number of features. You’ll also work with several richly featured real-world datasets, one containing a few thousand samples and one containing tens of thousands of samples. In the world of big data, datasets commonly have, millions and billions of samples, or even more.

There’s an enormous number of free and open datasets available for data science studies. Libraries like scikit-learn package up popular datasets for you to experiment with and provide mechanisms for loading datasets from various repositories (such as openml.org). Governments, businesses and other organizations worldwide offer datasets on a vast range of subjects. Between the text examples and the exercises and projects, you’ll work with many popular free datasets, using a variety of machine learning techniques.

We’ll use one of the simplest classification algorithms, k-nearest neighbors, to analyze the Digits dataset bundled with scikit-learn. Classification algorithms predict the discrete classes (categories) to which samples belong. Binary classification uses two classes, such as “spam” or “not spam” in an email classification application. Multi-classification uses more than two classes, such as the 10 classes, 0 through 9, in the Digits dataset. A classification scheme looking at movie descriptions might try to classify them as “action,” “adventure,” “fantasy,” “romance,” “history” and the like.

Regression models predict a continuous output, such as the predicted temperature output in the weather time series analysis from Chapter 10’s Intro to Data Science section. In this chapter, we’ll revisit that simple linear regression example, this time implementing it using scikit-learn’s LinearRegression estimator. Next, we use a LinearRegression estimator to perform multiple linear regression with the California Housing dataset that’s bundled with scikit-learn. We’ll predict the median house value of a U. S. census block of homes, considering eight features per block, such as the average number of rooms, median house age, average number of bedrooms and median income. The LinearRegression estimator, by default, uses all the numerical features in a dataset to make more sophisticated predictions than you can with a single-feature simple linear regression.

Next, we’ll introduce unsupervised machine learning with clustering algorithms. We’ll use dimensionality reduction (with scikit-learn’s TSNE estimator) to compress the Digits dataset’s 64 features down to two for visualization purposes. This will enable us to see how nicely the Digits data “cluster up.” This dataset contains handwritten digits like those the post office’s computers must recognize to route each letter to its designated zip code. This is a challenging computer-vision problem, given that each person’s handwriting is unique. Yet, we’ll build this clustering model with just a few lines of code and achieve impressive results. And we’ll do this without having to understand the inner workings of the clustering algorithm. This is the beauty of object-based programming. We’ll see this kind of convenient object-based programming again in the next chapter, where we’ll build powerful deep learning models using the open source Keras library.

We’ll present the simplest unsupervised machine-learning algorithm, k-means clustering, and use it on the Iris dataset that’s also bundled with scikit-learn. We’ll use dimensionality reduction (with scikit-learn’s PCA estimator) to compress the Iris dataset’s four features to two for visualization purposes. We’ll show the clustering of the three Iris species in the dataset and graph each cluster’s centroid, which is the cluster’s center point. Finally, we’ll run multiple clustering estimators to compare their ability to divide the Iris dataset’s samples effectively into three clusters.

You normally specify the desired number of clusters, k. K-means works through the data trying to divide it into that many clusters. As with many machine learning algorithms, k-means is iterative and gradually zeros in on the clusters to match the number you specify.

K-means clustering can find similarities in unlabeled data. This can ultimately help with assigning labels to that data so that supervised learning estimators can then process it. Given that it’s tedious and error-prone for humans to have to assign labels to unlabeled data, and given that the vast majority of the world’s data is unlabeled, unsupervised machine learning is an important tool.

The amount of data that’s available today is already enormous and continues to grow exponentially. The data produced in the world in the last few years equals the amount produced up to that point since the dawn of civilization. We commonly talk about big data, but “big” may not be a strong enough term to describe truly how huge data is getting.

People used to say “I’m drowning in data and I don’t know what to do with it.” With machine learning, we now say, “Flood me with big data so I can use machine-learning technology to extract insights and make predictions from it.”

This is occurring at a time when computing power is exploding and computer memory and secondary storage are exploding in capacity while costs dramatically decline. All of this enables us to think differently about the solution approaches. We now can program computers to learn from data, and lots of it. It’s now all about predicting from data.