preface

  acknowledgments

  about this book

  about the author

  about the cover illustration

  

Part 1 Deep learning foundation

  1 Welcome to computer vision

Computer vision

What is visual perception?

Vision systems

Sensing devices

Interpreting devices

Applications of computer vision

Image classification

Object detection and localization

Generating art (style transfer)

Creating images

Face recognition

Image recommendation system

Computer vision pipeline: The big picture

Image input

Image as functions

How computers see images

Color images

Image preprocessing

Converting color images to grayscale to reduce computation complexity

Feature extraction

What is a feature in computer vision?

What makes a good (useful) feature?

Extracting features (handcrafted vs. automatic extracting)

Classifier learning algorithm

  2 Deep learning and neural networks

Understanding perceptrons

What is a perceptron?

How does the perceptron learn?

Is one neuron enough to solve complex problems?

Multilayer perceptrons

Multilayer perceptron architecture

What are hidden layers?

How many layers, and how many nodes in each layer?

Some takeaways from this section

Activation functions

Linear transfer function

Heaviside step function (binary classifier)

Sigmoid/logistic function

Softmax function

Hyperbolic tangent function (tanh)

Rectified linear unit

Leaky ReLU

The feedforward process

Feedforward calculations

Feature learning

Error functions

What is the error function?

Why do we need an error function?

Error is always positive

Mean square error

Cross-entropy

A final note on errors and weights

Optimization algorithms

What is optimization?

Batch gradient descent

Stochastic gradient descent

Mini-batch gradient descent

Gradient descent takeaways

Backpropagation

What is backpropagation?

Backpropagation takeaways

  3 Convolutional neural networks

Image classification using MLP

Input layer

Hidden layers

Output layer

Putting it all together

Drawbacks of MLPs for processing images

CNN architecture

The big picture

A closer look at feature extraction

A closer look at classification

Basic components of a CNN

Convolutional layers

Pooling layers or subsampling

Fully connected layers

Image classification using CNNs

Building the model architecture

Number of parameters (weights)

Adding dropout layers to avoid overfitting

What is overfitting?

What is a dropout layer?

Why do we need dropout layers?

Where does the dropout layer go in the CNN architecture?

Convolution over color images (3D images)

How do we perform a convolution on a color image?

What happens to the computational complexity?

Project: Image classification for color images

  4 Structuring DL projects and hyperparameter tuning

Defining performance metrics

Is accuracy the best metric for evaluating a model?

Confusion matrix

Precision and recall

F-score

Designing a baseline model

Getting your data ready for training

Splitting your data for train/validation/test

Data preprocessing

Evaluating the model and interpreting its performance

Diagnosing overfitting and underfitting

Plotting the learning curves

Exercise: Building, training, and evaluating a network

Improving the network and tuning hyperparameters

Collecting more data vs. tuning hyperparameters

Parameters vs. hyperparameters

Neural network hyperparameters

Network architecture

Learning and optimization

Learning rate and decay schedule

A systematic approach to find the optimal learning rate

Learning rate decay and adaptive learning

Mini-batch size

Optimization algorithms

Gradient descent with momentum

Adam

Number of epochs and early stopping criteria

Early stopping

Regularization techniques to avoid overfitting

L2 regularization

Dropout layers

Data augmentation

Batch normalization

The covariate shift problem

Covariate shift in neural networks

How does batch normalization work?

Batch normalization implementation in Keras

Batch normalization recap

Project: Achieve high accuracy on image classification

Part 2 Image classification and detection

  5 Advanced CNN architectures

CNN design patterns

LeNet-5

LeNet architecture

LeNet-5 implementation in Keras

Setting up the learning hyperparameters

LeNet performance on the MNIST dataset

AlexNet

AlexNet architecture

Novel features of AlexNet

AlexNet implementation in Keras

Setting up the learning hyperparameters

AlexNet performance

VGGNet

Novel features of VGGNet

VGGNet configurations

Learning hyperparameters

VGGNet performance

Inception and GoogLeNet

Novel features of Inception

Inception module: Naive version

Inception module with dimensionality reduction

Inception architecture

GoogLeNet in Keras

Learning hyperparameters

Inception performance on the CIFAR dataset

ResNet

Novel features of ResNet

Residual blocks

ResNet implementation in Keras

Learning hyperparameters

ResNet performance on the CIFAR dataset

  6 Transfer learning

What problems does transfer learning solve?

What is transfer learning?

How transfer learning works

How do neural networks learn features?

Transferability of features extracted at later layers

Transfer learning approaches

Using a pretrained network as a classifier

Using a pretrained network as a feature extractor

Fine-tuning

Choosing the appropriate level of transfer learning

Scenario 1: Target dataset is small and similar to the source dataset

Scenario 2: Target dataset is large and similar to the source dataset

Scenario 3: Target dataset is small and different from the source dataset

Scenario 4: Target dataset is large and different from the source dataset

Recap of the transfer learning scenarios

Open source datasets

MNIST

Fashion-MNIST

CIFAR

ImageNet

MS COCO

Google Open Images

Kaggle

Project 1: A pretrained network as a feature extractor

Project 2: Fine-tuning

  7 Object detection with R-CNN, SSD, and YOLO

General object detection framework

Region proposals

Network predictions

Non-maximum suppression (NMS)

Object-detector evaluation metrics

Region-based convolutional neural networks (R-CNNs)

R-CNN

Fast R-CNN

Faster R-CNN

Recap of the R-CNN family

Single-shot detector (SSD)

High-level SSD architecture

Base network

Multi-scale feature layers

Non-maximum suppression

You only look once (YOLO)

How YOLOv3 works

YOLOv3 architecture

Project: Train an SSD network in a self-driving car application

Step 1: Build the model

Step 2: Model configuration

Step 3: Create the model

Step 4: Load the data

Step 5: Train the model

Step 6: Visualize the loss

Step 7: Make predictions

Part 3 Generative models and visual embeddings

  8 Generative adversarial networks (GANs)

GAN architecture

Deep convolutional GANs (DCGANs)

The discriminator model

The generator model

Training the GAN

GAN minimax function

Evaluating GAN models

Inception score

Fréchet inception distance (FID)

Which evaluation scheme to use

Popular GAN applications

Text-to-photo synthesis

Image-to-image translation (Pix2Pix GAN)

Image super-resolution GAN (SRGAN)

Ready to get your hands dirty?

Project: Building your own GAN

  9 DeepDream and neural style transfer

How convolutional neural networks see the world

Revisiting how neural networks work

Visualizing CNN features

Implementing a feature visualizer

DeepDream

How the DeepDream algorithm works

DeepDream implementation in Keras

Neural style transfer

Content loss

Style loss

Total variance loss

Network training

10 Visual embeddings

Applications of visual embeddings

Face recognition

Image recommendation systems

Object re-identification

Learning embedding

Loss functions

Problem setup and formalization

Cross-entropy loss

Contrastive loss

Triplet loss

Naive implementation and runtime analysis of losses

Mining informative data

Dataloader

Informative data mining: Finding useful triplets

Batch all (BA)

Batch hard (BH)

Batch weighted (BW)

Batch sample (BS)

Project: Train an embedding network

Fashion: Get me items similar to this

Vehicle re-identification

Implementation

Testing a trained model

Pushing the boundaries of current accuracy

  

appendix A. Getting set up

  

  index