Home Page Icon
Home Page
Table of Contents for
Deep Learning
Close
Deep Learning
by
Deep Learning
Title Page
Copyright
Dedication
About the Author
Acknowledgments
Introduction
Who This Book Is For
This Book Has No Complex Math and No Code
There Is Code, If You Want It
The Figures Are Available, Too!
Errata
About This Book
Part I: Foundational Ideas
Part II: Basic Machine Learning
Part III: Deep Learning Basics
Part IV: Beyond the Basics
Final Words
Part I: Foundational Ideas
Chapter 1: An Overview of Machine Learning
Expert Systems
Supervised Learning
Unsupervised Learning
Reinforcement Learning
Deep Learning
Summary
Chapter 2: Essential Statistics
Describing Randomness
Random Variables and Probability Distributions
Some Common Distributions
Continuous Distributions
Discrete Distributions
Collections of Random Values
Expected Value
Dependence
Independent and Identically Distributed Variables
Sampling and Replacement
Selection with Replacement
Selection Without Replacement
Bootstrapping
Covariance and Correlation
Covariance
Correlation
Statistics Don’t Tell Us Everything
High-Dimensional Spaces
Summary
Chapter 3: Measuring Performance
Different Types of Probability
Dart Throwing
Simple Probability
Conditional Probability
Joint Probability
Marginal Probability
Measuring Correctness
Classifying Samples
The Confusion Matrix
Characterizing Incorrect Predictions
Measuring Correct and Incorrect
Accuracy
Precision
Recall
Precision-Recall Tradeoff
Misleading Measures
f1 Score
About These Terms
Other Measures
Constructing a Confusion Matrix Correctly
Summary
Chapter 4: Bayes’ Rule
Frequentist and Bayesian Probability
The Frequentist Approach
The Bayesian Approach
Frequentists vs. Bayesians
Frequentist Coin Flipping
Bayesian Coin Flipping
A Motivating Example
Picturing the Coin Probabilities
Expressing Coin Flips as Probabilities
Bayes’ Rule
Discussion of Bayes’ Rule
Bayes’ Rule and Confusion Matrices
Repeating Bayes’ Rule
The Posterior-Prior Loop
The Bayes Loop in Action
Multiple Hypotheses
Summary
Chapter 5: Curves and Surfaces
The Nature of Functions
The Derivative
Maximums and Minimums
Tangent Lines
Finding Minimums and Maximums with Derivatives
The Gradient
Water, Gravity, and the Gradient
Finding Maximums and Minimums with Gradients
Saddle Points
Summary
Chapter 6: Information Theory
Surprise and Context
Understanding Surprise
Unpacking Context
Measuring Information
Adaptive Codes
Speaking Morse
Customizing Morse Code
Entropy
Cross Entropy
Two Adaptive Codes
Using the Codes
Cross Entropy in Practice
Kullback–Leibler Divergence
Summary
Part II: Basic Machine Learning
Chapter 7: Classification
Two-Dimensional Binary Classification
2D Multiclass Classification
Multiclass Classification
One-Versus-Rest
One-Versus-One
Clustering
The Curse of Dimensionality
Dimensionality and Density
High-Dimensional Weirdness
Summary
Chapter 8: Training and Testing
Training
Testing the Performance
Test Data
Validation Data
Cross-Validation
k-Fold Cross-Validation
Summary
Chapter 9: Overfitting and Underfitting
Finding a Good Fit
Overfitting
Underfitting
Detecting and Addressing Overfitting
Early Stopping
Regularization
Bias and Variance
Matching the Underlying Data
High Bias, Low Variance
Low Bias, High Variance
Comparing Curves
Fitting a Line with Bayes’ Rule
Summary
Chapter 10: Data Preparation
Basic Data Cleaning
The Importance of Consistency
Types of Data
One-Hot Encoding
Normalizing and Standardizing
Normalization
Standardization
Remembering the Transformation
Types of Transformations
Slice Processing
Samplewise Processing
Featurewise Processing
Elementwise Processing
Inverse Transformations
Information Leakage in Cross-Validation
Shrinking the Dataset
Feature Selection
Dimensionality Reduction
Principal Component Analysis
PCA for Simple Images
PCA for Real Images
Summary
Chapter 11: Classifiers
Types of Classifiers
k-Nearest Neighbors
Decision Trees
Using Decision Trees
Overfitting Trees
Splitting Nodes
Support Vector Machines
The Basic Algorithm
The SVM Kernel Trick
Naive Bayes
Comparing Classifiers
Summary
Chapter 12: Ensembles
Voting
Ensembles of Decision Trees
Bagging
Random Forests
Extra Trees
Boosting
Summary
Part III: Deep Learning Basics
Chapter 13: Neural Networks
Real Neurons
Artificial Neurons
The Perceptron
Modern Artificial Neurons
Drawing the Neurons
Feed-Forward Networks
Neural Network Graphs
Initializing the Weights
Deep Networks
Fully Connected Layers
Tensors
Preventing Network Collapse
Activation Functions
Straight-Line Functions
Step Functions
Piecewise Linear Functions
Smooth Functions
Activation Function Gallery
Comparing Activation Functions
Softmax
Summary
Chapter 14: Backpropagation
A High-Level Overview of Training
Punishing Error
A Slow Way to Learn
Gradient Descent
Getting Started
Backprop on a Tiny Neural Network
Finding Deltas for the Output Neurons
Using Deltas to Change Weights
Other Neuron Deltas
Backprop on a Larger Network
The Learning Rate
Building a Binary Classifier
Picking a Learning Rate
An Even Smaller Learning Rate
Summary
Chapter 15: Optimizers
Error as a 2D Curve
Adjusting the Learning Rate
Constant-Sized Updates
Changing the Learning Rate over Time
Decay Schedules
Updating Strategies
Batch Gradient Descent
Stochastic Gradient Descent
Mini-Batch Gradient Descent
Gradient Descent Variations
Momentum
Nesterov Momentum
Adagrad
Adadelta and RMSprop
Adam
Choosing an Optimizer
Regularization
Dropout
Batchnorm
Summary
PART IV: Beyond the Basics
Chapter 16: Convolutional Neural Networks
Introducing Convolution
Detecting Yellow
Weight Sharing
Larger Filters
Filters and Features
Padding
Multidimensional Convolution
Multiple Filters
Convolution Layers
1D Convolution
1×1 Convolutions
Changing Output Size
Pooling
Striding
Transposed Convolution
Hierarchies of Filters
Simplifying Assumptions
Finding Face Masks
Finding Eyes, Noses, and Mouths
Applying Our Filters
Summary
Chapter 17: Convnets in Practice
Categorizing Handwritten Digits
VGG16
Visualizing Filters, Part 1
Visualizing Filters, Part 2
Adversaries
Summary
Chapter 18: Autoencoders
Introduction to Encoding
Lossless and Lossy Encoding
Blending Representations
The Simplest Autoencoder
A Better Autoencoder
Exploring the Autoencoder
A Closer Look at the Latent Variables
The Parameter Space
Blending Latent Variables
Predicting from Novel Input
Convolutional Autoencoders
Blending Latent Variables
Predicting from Novel Input
Denoising
Variational Autoencoders
Distribution of Latent Variables
Variational Autoencoder Structure
Exploring the VAE
Working with the MNIST Samples
Working with Two Latent Variables
Producing New Input
Summary
Chapter 19: Recurrent Neural Networks
Working with Language
Common Natural Language Processing Tasks
Transforming Text into Numbers
Fine-Tuning and Downstream Networks
Fully Connected Prediction
Testing Our Network
Why Our Network Failed
Recurrent Neural Networks
Introducing State
Rolling Up Our Diagram
Recurrent Cells in Action
Training a Recurrent Neural Network
Long Short-Term Memory and Gated Recurrent Networks
Using Recurrent Neural Networks
Working with Sunspot Data
Generating Text
Different Architectures
Seq2Seq
Summary
Chapter 20: Attention and Transformers
Embedding
Embedding Words
ELMo
Attention
A Motivating Analogy
Self-Attention
Q/KV Attention
Multi-Head Attention
Layer Icons
Transformers
Skip Connections
Norm-Add
Positional Encoding
Assembling a Transformer
Transformers in Action
BERT and GPT-2
BERT
GPT-2
Generators Discussion
Data Poisoning
Summary
Chapter 21: Reinforcement Learning
Basic Ideas
Learning a New Game
The Structure of Reinforcement Learning
Step 1: The Agent Selects an Action
Step 2: The Environment Responds
Step 3: The Agent Updates Itself
Back to the Big Picture
Understanding Rewards
Flippers
L-Learning
The Basics
The L-Learning Algorithm
Testing Our Algorithm
Handling Unpredictability
Q-Learning
Q-Values and Updates
Q-Learning Policy
Putting It All Together
The Elephant in the Room
Q-learning in Action
SARSA
The Algorithm
SARSA in Action
Comparing Q-Learning and SARSA
The Big Picture
Summary
Chapter 22: Generative Adversarial Networks
Forging Money
Learning from Experience
Forging with Neural Networks
A Learning Round
Why Adversarial?
Implementing GANs
The Discriminator
The Generator
Training the GAN
GANs in Action
Building a Discriminator and Generator
Training Our Network
Testing Our Network
DCGANs
Challenges
Using Big Samples
Modal Collapse
Training with Generated Data
Summary
Chapter 23: Creative Applications
Deep Dreaming
Stimulating Filters
Running Deep Dreaming
Neural Style Transfer
Representing Style
Representing Content
Style and Content Together
Running Style Transfer
Generating More of This Book
Summary
Final Thoughts
References
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
Chapter 13
Chapter 14
Chapter 15
Chapter 16
Chapter 17
Chapter 18
Chapter 19
Chapter 20
Chapter 21
Chapter 22
Chapter 23
Image Credits
Chapter 1
Chapter 10
Chapter 16
Chapter 17
Chapter 18
Chapter 23
Index
PART V: Bonus Chapters
Chapter B1: SciKit-Learn
Python Conventions and Libraries
Estimators
Creation
Learning with fit()
Predicting with predict()
Using decision_function() and predict_proba()
Clustering
Transformations
Inverse Transformations
Data Refinement
Ensembles
Automation
Cross-Validation
Hyperparameter Searching
Exhaustive Grid Search
Random Grid Search
Pipelines
Looking at the Decision Boundary
Applying Pipelined Transformations
Datasets
Utilities
Wrapping Up
References
Chapter B2: Keras Part 1
The Structure of This Chapter
Libraries, Programming, and Debugging
Versions and Programming Style
Python Programming and Debugging
Running Externally
A Workaround Note
Overview
Tensors and Arrays
Setting Up Keras
Shapes of Tensors Holding Images
GPUs and Other Accelerators
Getting Started
Hello, World
Preparing the Data
Reshaping
Loading the Data
Looking at the Data
Train-test Splitting
Fixing the Data Type
Normalizing the Data
Fixing the Labels
Pre-Processing All in One Place
Making the Model
Turning Grids into Lists
Creating the Model
Compiling the Model
Model Creation Summary
Training the Model
Training and Using Our Model
Looking at the Output
Prediction
Analysis of Training History
Saving and Loading
Saving Everything in One File
Saving Just the Weights
Saving Just the Architecture
Using Pre-Trained Models
Saving the Pre-Processing Steps
Callbacks
Checkpoints
Learning Rate
Early Stopping
Wrapping Up
References
Image Credits
Chapter B3: Keras Part 2
Improving the Model
Counting Up Hyperparameters
Changing One Hyperparameter
Other Ways to Improve
Adding Another Dense Layer
Less Is More
Adding Dropout
Observations
Using Scikit-Learn
Keras Wrappers
Cross-Validation
Cross-Validation with Normalization
Hyperparameter Searching
Convolution Networks
Utility Layers
Preparing the Data for A CNN
Convolution Layers
Using Convolution for MNIST
Patterns
Image Data Augmentation
Synthetic Data
Parameter Searching for Convnets
RNNs
Generating Sequence Data
RNN Data Preparation
Building, Compiling, and Running the RNN
Analyzing RNN Performance
A More Complex Dataset
Deep RNNS
The Value of More Data
Returning Sequences
Stateful RNNs
Time-Distributed Layers
Generating Text
The Functional API
Input Layers
Making A Functional Model
Summary
References
Image Credits
Search in book...
Toggle Font Controls
Playlists
Add To
Create new playlist
Name your new playlist
Playlist description (optional)
Cancel
Create playlist
Sign In
Email address
Password
Forgot Password?
Create account
Login
or
Continue with Facebook
Continue with Google
Sign Up
Full Name
Email address
Confirm Email Address
Password
Login
Create account
or
Continue with Facebook
Continue with Google
Next
Next Chapter
Deep Learning
Add Highlight
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click
here
login for view all page.
Day Mode
Cloud Mode
Night Mode
Reset