Contents
1.2 Scope, Terminology, Prediction, and Data
1.2.2 Target Values and Predictions
1.3 Putting the Machine in Machine Learning
1.4 Examples of Learning Systems
1.4.1 Predicting Categories: Examples of Classifiers
1.4.2 Predicting Values: Examples of Regressors
1.5 Evaluating Learning Systems
1.6 A Process for Building Learning Systems
1.7 Assumptions and Reality of Learning
2.2 The Need for Mathematical Language
2.3 Our Software for Tackling Machine Learning
2.5 Linear Combinations, Weighted Sums, and Dot Products
2.6 A Geometric View: Points in Space
2.7 Notation and the Plus-One Trick
2.8 Getting Groovy, Breaking the Straight-Jacket, and Nonlinearity
2.9 NumPy versus “All the Maths”
3 Predicting Categories: Getting Started with Classification
3.2 A Simple Classification Dataset
3.3 Training and Testing: Don’t Teach to the Test
3.4 Evaluation: Grading the Exam
3.5 Simple Classifier #1: Nearest Neighbors, Long Distance Relationships, and Assumptions
3.5.4 k-NN, Parameters, and Nonparametric Methods
3.5.5 Building a k-NN Classification Model
3.6 Simple Classifier #2: Naive Bayes, Probability, and Broken Promises
3.7 Simplistic Evaluation of Classifiers
3.7.2 Resource Utilization in Classification
3.7.3 Stand-Alone Resource Evaluation
3.8.1 Sophomore Warning: Limitations and Open Issues
4 Predicting Numerical Values: Getting Started with Regression
4.1 A Simple Regression Dataset
4.2 Nearest-Neighbors Regression and Summary Statistics
4.2.1 Measures of Center: Median and Mean
4.2.2 Building a k-NN Regression Model
4.3 Linear Regression and Errors
4.3.1 No Flat Earth: Why We Need Slope
4.3.3 Performing Linear Regression
4.4 Optimization: Picking the Best Answer
4.4.5 Application to Linear Regression
4.5 Simple Evaluation and Comparison of Regressors
4.5.3 Resource Utilization in Regression
4.6.1 Limitations and Open Issues
5 Evaluating and Comparing Learners
5.1 Evaluation and Why Less Is More
5.2 Terminology for Learning Phases
5.2.2 More Technically Speaking . . .
5.3 Major Tom, There’s Something Wrong: Overfitting and Underfitting
5.3.1 Synthetic Data and Linear Regression
5.3.2 Manually Manipulating Model Complexity
5.3.3 Goldilocks: Visualizing Overfitting, Underfitting, and “Just Right”
5.3.5 Take-Home Notes on Overfitting
5.5 (Re)Sampling: Making More from Less
5.5.3 Repeated Train-Test Splits
5.5.4 A Better Way and Shuffling
5.5.5 Leave-One-Out Cross-Validation
5.6 Break-It-Down: Deconstructing Error into Bias and Variance
5.6.5 Examples of Bias-Variance Tradeoffs
5.7 Graphical Evaluation and Comparison
5.7.1 Learning Curves: How Much Data Do We Need?
5.8 Comparing Learners with Cross-Validation
6.2 Beyond Accuracy: Metrics for Classification
6.2.1 Eliminating Confusion from the Confusion Matrix
6.2.3 Metrics from the Confusion Matrix
6.2.4 Coding the Confusion Matrix
6.2.5 Dealing with Multiple Classes: Multiclass Averaging
6.3.3 AUC: Area-Under-the-(ROC)-Curve
6.3.4 Multiclass Learners, One-versus-Rest, and ROC
6.4 Another Take on Multiclass: One-versus-One
6.4.1 Multiclass AUC Part Two: The Quest for a Single Value
6.5.1 A Note on Precision-Recall Tradeoff
6.5.2 Constructing a Precision-Recall Curve
6.6 Cumulative Response and Lift Curves
6.7 More Sophisticated Evaluation of Classifiers: Take Two
6.7.2 A Novel Multiclass Problem
7.2 Additional Measures for Regression
7.2.1 Creating Our Own Evaluation Metric
7.2.2 Other Built-in Regression Metrics
7.4 A First Look at Standardization
7.5 Evaluating Regressors in a More Sophisticated Way: Take Two
7.5.1 Cross-Validated Results on Multiple Metrics
7.5.2 Summarizing Cross-Validated Results
III More Methods and Fundamentals
8.2.1 Tree-Building Algorithms
8.2.2 Let’s Go: Decision Tree Time
8.2.3 Bias and Variance in Decision Trees
8.3 Support Vector Classifiers
8.3.2 Bias and Variance in SVCs
8.4.2 Probabilities, Odds, and Log-Odds
8.4.3 Just Do It: Logistic Regression Edition
8.4.4 A Logistic Regression: A Space Oddity
8.6 Assumptions, Biases, and Classifiers
8.7 Comparison of Classifiers: Take Three
9.1 Linear Regression in the Penalty Box: Regularization
9.1.1 Performing Regularized Regression
9.2.2 From Linear Regression to Regularized Regression to Support Vector Regression
9.3 Piecewise Constant Regression
9.3.1 Implementing a Piecewise Constant Regressor
9.3.2 General Notes on Implementing Models
9.4.1 Performing Regression with Trees
9.5 Comparison of Regressors: Take Three
10 Manual Feature Engineering: Manipulating Data for Fun and Profit
10.1 Feature Engineering Terminology and Motivation
10.1.2 When Does Engineering Happen?
10.1.3 How Does Feature Engineering Occur?
10.2 Feature Selection and Data Reduction: Taking out the Trash
10.5.1 Another Way to Code and the Curious Case of the Missing Intercept
10.6 Relationships and Interactions
10.6.1 Manual Feature Construction
10.6.3 Adding Features with Transformers
10.7.1 Manipulating the Input Space
10.7.2 Manipulating the Target
11 Tuning Hyperparameters and Pipelines
11.1 Models, Parameters, Hyperparameters
11.2.1 A Note on Computer Science and Learning Terminology
11.2.2 An Example of Complete Search
11.2.3 Using Randomness to Search for a Needle in a Haystack
11.3 Down the Recursive Rabbit Hole: Nested Cross-Validation
11.3.1 Cross-Validation, Redux
11.3.3 Cross-Validation Nested within Cross-Validation
11.4.2 A More Complex Pipeline
11.5 Pipelines and Tuning Together
12.3 Bagging and Random Forests
12.3.2 From Bootstrapping to Bagging
12.3.3 Through the Random Forest
12.5 Comparing the Tree-Ensemble Methods
13 Models That Engineer Features for Us
13.1.1 Single-Step Filtering with Metric-Based Feature Selection
13.1.2 Model-Based Feature Selection
13.1.3 Integrating Feature Selection with a Learning Pipeline
13.2 Feature Construction with Kernels
13.2.3 Kernel Methods and Kernel Options
13.2.5 Take-Home Notes on SVM and an Example
13.3 Principal Components Analysis: An Unsupervised Technique
13.3.2 Finding a Different Best Line
13.3.5 A Finale: Comments on General PCA
13.3.6 Kernel PCA and Manifold Methods
14 Feature Engineering for Domains: Domain-Specific Learning
14.1.2 Example of Text Learning
14.3.4 Complete Code of BoVW Transformer
15 Connections, Extensions, and Further Directions
15.2 Linear Regression from Raw Materials
15.2.1 A Graphical View of Linear Regression
15.3 Building Logistic Regression from Raw Materials
15.3.1 Logistic Regression with Zero-One Coding
15.3.2 Logistic Regression with Plus-One Minus-One Coding
15.3.3 A Graphical View of Logistic Regression
15.5.1 A NN View of Linear Regression
15.5.2 A NN View of Logistic Regression
15.5.3 Beyond Basic Neural Networks
15.6 Probabilistic Graphical Models
15.6.2 A PGM View of Linear Regression