Artificial Intelligence and Machine Learning Fundamentals

Develop real-world applications powered by the latest AI advances

Zsolt Nagy

Artificial Intelligence and Machine Learning Fundamentals

Copyright © 2018 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Author: Zsolt Nagy

Reviewer: Richard Ackon

Managing Editor: Steffi Monteiro

Acquisitions Editor: Koushik Sen

Production Editor: Nitesh Thakur

Editorial Board: David Barnes, Ewan Buckingham, Simon Cox, Manasa Kumar, Alex Mazonowicz, Douglas Paterson, Dominic Pereira, Shiny Poojary, Saman Siddiqui, Erol Staveley, Ankita Thakur, and Mohita Vyas.

First published: December 2018

Production reference: 1071218

Published by Packt Publishing Ltd.

Livery Place, 35 Livery Street

Birmingham B3 2PB, UK

ISBN 978-1-78980-165-1

Table of Contents

Preface   i

Principles of Artificial Intelligence   1

Introduction   2

How does AI Solve Real World Problems?   2

Diversity of Disciplines   3

Fields and Applications of Artificial Intelligence   4

Simulating Intelligence – The Turing Test   7

AI Tools and Learning Models   7

Classification and Prediction   8

Learning Models   8

The Role of Python in Artificial Intelligence   9

Why is Python Dominant in Machine Learning, Data Science, and AI?    9

Anaconda in Python   10

Python Libraries for Artificial Intelligence   11

A Brief Introduction to the NumPy Library   12

Exercise 1: Matrix Operations Using NumPy   14

Python for Game AI   16

Intelligent Agents in Games   16

Breadth First Search and Depth First Search   17

Exploring the State Space of a Game   20

Exercise 2: Estimating the Number of Possible States in Tic-Tac-Toe Game   22

Exercise 3: Creating an AI Randomly   23

Activity 1: Generating All Possible Sequences of Steps in a Tic-Tac-Toe Game   26

Summary   27

AI with Search Techniques and Games   29

Introduction   30

Exercise 4: Teaching the Agent to Win   30

Activity 2: Teaching the Agent to Realize Situations When It Defends Against Losses   32

Activity 3: Fixing the First and Second Moves of the AI to Make it Invincible   33

Heuristics   35

Uninformed and Informed Search   35

Creating Heuristics   35

Admissible and Non-Admissible Heuristics   36

Heuristic Evaluation   36

Exercise 5: Tic-Tac-Toe Static Evaluation with a Heuristic Function   39

Using Heuristics for an Informed Search   41

Types of Heuristics   41

Pathfinding with the A* Algorithm   42

Exercise 6: Finding the Shortest Path to Reach a Goal   44

Exercise 7: Finding the Shortest Path Using BFS   45

Introducing the A* Algorithm   47

A* Search in Practice Using the simpleai Library   60

Game AI with the Minmax Algorithm and Alpha-Beta Pruning    63

Search Algorithms for Turn-Based Multiplayer Games   64

The Minmax Algorithm   65

Optimizing the Minmax Algorithm with Alpha-Beta Pruning   70

DRYing up the Minmax Algorithm – The NegaMax Algorithm   73

Using the EasyAI Library    74

Activity 4: Connect Four   77

Summary   78

Regression   81

Introduction   82

Linear Regression with One Variable   82

What Is Regression?   83

Features and Labels   86

Feature Scaling   87

Cross-Validation with Training and Test Data   89

Fitting a Model on Data with scikit-learn    90

Linear Regression Using NumPy Arrays   92

Fitting a Model Using NumPy Polyfit   97

Predicting Values with Linear Regression   104

Activity 5: Predicting Population   105

Linear Regression with Multiple Variables   106

Multiple Linear Regression   106

The Process of Linear Regression   107

Importing Data from Data Sources   108

Loading Stock Prices with Yahoo Finance   108

Loading Files with pandas   109

Loading Stock Prices with Quandl   109

Exercise 8: Using Quandl to Load Stock Prices   110

Preparing Data for Prediction   111

Performing and Validating Linear Regression   116

Predicting the Future   118

Polynomial and Support Vector Regression   122

Polynomial Regression with One Variable   123

Exercise 9: 1st, 2nd, and 3rd Degree Polynomial Regression   124

Polynomial Regression with Multiple Variables   127

Support Vector Regression   129

Support Vector Machines with a 3 Degree Polynomial Kernel   132

Activity 6: Stock Price Prediction with Quadratic and Cubic Linear Polynomial Regression with Multiple Variables   133

Summary   134

Classification   137

Introduction   138

The Fundamentals of Classification   138

Exercise 10: Loading Datasets   139

Data Preprocessing   142

Exercise 11: Pre-Processing Data   145

Minmax Scaling of the Goal Column   147

Identifying Features and Labels   148

Cross-Validation with scikit-learn   148

Activity 7: Preparing Credit Data for Classification   149

The k-nearest neighbor Classifier   149

Introducing the K-Nearest Neighbor Algorithm   150

Distance Functions   150

Exercise 12: Illustrating the K-nearest Neighbor Classifier Algorithm   152

Exercise 13: k-nearest Neighbor Classification in scikit-learn   155

Exercise 14: Prediction with the k-nearest neighbors classifier   156

Parameterization of the k-nearest neighbor Classifier in scikit-learn   156

Activity 8: Increasing the Accuracy of Credit Scoring   157

Classification with Support Vector Machines   157

What are Support Vector Machine Classifiers?   158

Understanding Support Vector Machines   159

Support Vector Machines in scikit-learn   162

Parameters of the scikit-learn SVM   163

Activity 9: Support Vector Machine Optimization in scikit-learn   163

Summary   164

Using Trees for Predictive Analysis   167

Introduction to Decision Trees   168

Entropy   172

Exercise 15: Calculating the Entropy   174

Information Gain   176

Gini Impurity   177

Exit Condition   178

Building Decision Tree Classifiers using scikit-learn   179

Evaluating the Performance of Classifiers   180

Exercise 16: Precision and Recall   182

Exercise 17: Calculating the F1 Score   183

Confusion Matrix   186

Exercise 18: Confusion Matrix   186

Activity 10: Car Data Classification   188

Random Forest Classifier   189

Constructing a Random Forest   190

Random Forest Classification Using scikit-learn   191

Parameterization of the random forest classifier   192

Feature Importance   192

Extremely Randomized Trees   193

Activity 11: Random Forest Classification for Your Car Rental Company   193

Summary   194

Clustering   197

Introduction to Clustering   198

Defining the Clustering Problem   198

Clustering Approaches   201

Clustering Algorithms Supported by scikit-learn   201

The k-means Algorithm   202

Exercise 19: k-means in scikit-learn   203

Parameterization of the k-means Algorithm in scikit-learn   208

Exercise 20: Retrieving the Center Points and the Labels   208

k-means Clustering of Sales Data   209

Activity 12: k-means Clustering of Sales Data   209

Mean Shift Algorithm   210

Exercise 21: Illustrating Mean Shift in 2D   211

Mean Shift Algorithm in scikit-learn   216

Image Processing in Python   218

Activity 13: Shape Recognition with the Mean Shift Algorithm   220

Summary   221

Deep Learning with Neural Networks   223

Introduction    224

TensorFlow for Python   224

Installing TensorFlow in the Anaconda Navigator   225

TensorFlow Operations   226

Exercise 22: Using Basic Operations and TensorFlow constants   226

Placeholders and Variables   227

Global Variables Initializer   228

Introduction to Neural Networks   229

Biases   233

Use Cases for Artificial Neural Networks   234

Activation Functions   235

Exercise 23: Activation Functions   238

Forward and Backward Propagation   240

Configuring a Neural Network   241

Importing the TensorFlow Digit Dataset   241

Modeling Features and Labels   242

TensorFlow Modeling for Multiple Labels   244

Optimizing the Variables   245

Training the TensorFlow Model   247

Using the Model for Prediction   247

Testing the Model   248

Randomizing the Sample Size   249

Activity 14: Written Digit Detection   250

Deep Learning   251

Adding Layers   251

Convolutional Neural Networks   252

Activity 15: Written Digit Detection with Deep Learning   253

Summary   254

Appendix   257