Home Page Icon
Home Page
Table of Contents for
Grokking Deep Learning
Close
Grokking Deep Learning
by Andrew W. Trask
Grokking Deep Learning
Copyright
Dedication
Brief Table of Contents
Table of Contents
Preface
Acknowledgments
About this book
Who should read this book
Roadmap
About the Code conventions and downloads
Book forum
About the author
Chapter 1. Introducing deep learning: why you should learn it
Welcome to Grokking Deep Learning
Why you should learn deep learning
Will this be difficult to learn?
Why you should read this book
What you need to get started
You’ll probably need some Python knowledge
Summary
Chapter 2. Fundamental concepts: how do machines learn?
What is deep learning?
What is machine learning?
Supervised machine learning
Unsupervised machine learning
Parametric vs. nonparametric learning
Supervised parametric learning
Unsupervised parametric learning
Nonparametric learning
Summary
Chapter 3. Introduction to neural prediction: forward propagation
Step 1: Predict
A simple neural network making a prediction
What is a neural network?
What does this neural network do?
Making a prediction with multiple inputs
Multiple inputs: What does this neural network do?
Multiple inputs: Complete runnable code
Making a prediction with multiple outputs
Predicting with multiple inputs and outputs
Multiple inputs and outputs: How does it work?
Predicting on predictions
A quick primer on NumPy
Summary
Chapter 4. Introduction to neural learning: gradient descent
Predict, compare, and learn
Compare
Learn
Compare: Does your network make good predictions?
Why measure error?
What’s the simplest form of neural learning?
Hot and cold learning
Characteristics of hot and cold learning
Calculating both direction and amount from error
One iteration of gradient descent
Learning is just reducing error
Let’s watch several steps of learning
Why does this work? What is weight_delta, really?
Tunnel vision on one concept
A box with rods poking out of it
Derivatives: Take two
What you really need to know
What you don’t really need to know
How to use a derivative to learn
Look familiar?
Breaking gradient descent
Visualizing the overcorrections
Divergence
Introducing alpha
Alpha in code
Memorizing
Chapter 5. Learning multiple weights at a time: generalizing gradient descent
Gradient descent learning with multiple inputs
Gradient descent with multiple inputs explained
Let’s watch several steps of learning
Freezing one weight: What does it do?
Gradient descent learning with multiple outputs
Gradient descent with multiple inputs and outputs
What do these weights learn?
Visualizing weight values
Visualizing dot products (weighted sums)
Summary
Chapter 6. Building your first deep neural network: introduction to backpropagation
The streetlight problem
Preparing the data
Matrices and the matrix relationship
Creating a matrix or two in Python
Building a neural network
Learning the whole dataset
Full, batch, and stochastic gradient descent
Neural networks learn correlation
Up and down pressure
Edge case: Overfitting
Edge case: Conflicting pressure
Learning indirect correlation
Creating correlation
Stacking neural networks: A review
Backpropagation: Long-distance error attribution
Backpropagation: Why does this work?
Linear vs. nonlinear
Why the neural network still doesn’t work
The secret to sometimes correlation
A quick break
Your first deep neural network
Backpropagation in code
One iteration of backpropagation
Putting it all together
Why do deep networks matter?
Chapter 7. How to picture neural networks: in your head and on paper
It’s time to simplify
Correlation summarization
The previously overcomplicated visualization
The simplified visualization
Simplifying even further
Let’s see this network predict
Visualizing using letters instead of pictures
Linking the variables
Everything side by side
The importance of visualization tools
Chapter 8. Learning signal and ignoring noise: introduction to regularization and batching
Three-layer network on MNIST
Well, that was easy
Memorization vs. generalization
Overfitting in neural networks
Where overfitting comes from
The simplest regularization: Early stopping
Industry standard regularization: Dropout
Why dropout works: Ensembling works
Dropout in code
Dropout evaluated on MNIST
Batch gradient descent
Summary
Chapter 9. Modeling probabilities and nonlinearities: activation functions
What is an activation function?
Standard hidden-layer activation functions
Standard output layer activation functions
The core issue: Inputs have similarity
softmax computation
Activation installation instructions
Multiplying delta by the slope
Converting output to slope (derivative)
Upgrading the MNIST network
Chapter 10. Neural learning about edges and corners: intro to convolutional neural networks
Reusing weights in multiple places
The convolutional layer
A simple implementation in NumPy
Summary
Chapter 11. Neural networks that understand language: king – man + woman == ?
What does it mean to understand language?
Natural language processing (NLP)
Supervised NLP
IMDB movie reviews dataset
Capturing word correlation in input data
Predicting movie reviews
Intro to an embedding layer
Interpreting the output
Neural architecture
Comparing word embeddings
What is the meaning of a neuron?
Filling in the blank
Meaning is derived from loss
King – Man + Woman ~= Queen
Word analogies
Summary
Chapter 12. Neural networks that write like Shakespeare: recurrent layers for variable-length data
The challenge of arbitrary length
Do comparisons really matter?
The surprising power of averaged word vectors
How is information stored in these embeddings?
How does a neural network use embeddings?
The limitations of bag-of-words vectors
Using identity vectors to sum word embeddings
Matrices that change absolutely nothing
Learning the transition matrices
Learning to create useful sentence vectors
Forward propagation in Python
How do you backpropagate into this?
Let’s train it!
Setting things up
Forward propagation with arbitrary length
Backpropagation with arbitrary length
Weight update with arbitrary length
Execution and output analysis
Summary
Chapter 13. Introducing automatic optimization: let’s build a deep learning framework
What is a deep learning framework?
Introduction to tensors
Introduction to automatic gradient computation (autograd)
A quick checkpoint
Tensors that are used multiple times
Upgrading autograd to support multiuse tensors
How does addition backpropagation work?
Adding support for negation
Adding support for additional functions
Using autograd to train a neural network
Adding automatic optimization
Adding support for layer types
Layers that contain layers
Loss-function layers
How to learn a framework
Nonlinearity layers
The embedding layer
Adding indexing to autograd
The embedding layer (revisited)
The cross-entropy layer
The recurrent neural network layer
Summary
Chapter 14. Learning to write like Shakespeare: long short-term memory
Character language modeling
The need for truncated backpropagation
Truncated backpropagation
A sample of the output
Vanishing and exploding gradients
A toy example of RNN backpropagation
Long short-term memory (LSTM) cells
Some intuition about LSTM gates
The long short-term memory layer
Upgrading the character language model
Training the LSTM character language model
Tuning the LSTM character language model
Summary
Chapter 15. Deep learning on unseen data: introducing federated learning
The problem of privacy in deep learning
Federated learning
Learning to detect spam
Let’s make it federated
Hacking into federated learning
Secure aggregation
Homomorphic encryption
Homomorphically encrypted federated learning
Summary
Chapter 16. Where to go from here: a brief guide
Congratulations!
Step 1: Start learning PyTorch
Step 2: Start another deep learning course
Step 3: Grab a mathy deep learning textbook
Step 4: Start a blog, and teach deep learning
Step 5: Twitter
Step 6: Implement academic papers
Step 7: Acquire access to a GPU (or many)
Step 8: Get paid to practice
Step 9: Join an open source project
Step 10: Develop your local community
Index
Search in book...
Toggle Font Controls
Playlists
Add To
Create new playlist
Name your new playlist
Playlist description (optional)
Cancel
Create playlist
Sign In
Email address
Password
Forgot Password?
Create account
Login
or
Continue with Facebook
Continue with Google
Sign Up
Full Name
Email address
Confirm Email Address
Password
Login
Create account
or
Continue with Facebook
Continue with Google
Prev
Previous Chapter
Cover
Next
Next Chapter
Copyright
Grokking Deep Learning
Andrew W. Trask
Add Highlight
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click
here
login for view all page.
Day Mode
Cloud Mode
Night Mode
Reset