Cover image for Deep Learning with PyTorch 1.x

Deep Learning with PyTorch 1.x

Author Laura Mitchell , Sri. Yogesh K. , Vishnu Subramanian
Release Date: 2019/11/01
Topic:
0%
255
Chapters
0-1
Hours read
0k
Total Words
    Start Reading Now    
       Add to Wishlist       
View table of contents

Book Description

Build and train neural network models with high speed and flexibility in text, vision, and advanced analytics using PyTorch 1.x

Key Features

  • Gain a thorough understanding of the PyTorch framework and learn to implement neural network architectures
  • Understand GPU computing to perform heavy deep learning computations using Python
  • Apply cutting-edge natural language processing (NLP) techniques to solve problems with textual data

Book Description

PyTorch is gaining the attention of deep learning researchers and data science professionals due to its accessibility and efficiency, along with the fact that it's more native to the Python way of development. This book will get you up and running with this cutting-edge deep learning library, effectively guiding you through implementing deep learning concepts.

In this second edition, you'll learn the fundamental aspects that power modern deep learning, and explore the new features of the PyTorch 1.x library. You'll understand how to solve real-world problems using CNNs, RNNs, and LSTMs, along with discovering state-of-the-art modern deep learning architectures, such as ResNet, DenseNet, and Inception. You'll then focus on applying neural networks to domains such as computer vision and NLP. Later chapters will demonstrate how to build, train, and scale a model with PyTorch and also cover complex neural networks such as GANs and autoencoders for producing text and images. In addition to this, you'll explore GPU computing and how it can be used to perform heavy computations. Finally, you'll learn how to work with deep learning-based architectures for transfer learning and reinforcement learning problems.

By the end of this book, you'll be able to confidently and easily implement deep learning applications in PyTorch.

What you will learn

  • Build text classification and language modeling systems using neural networks
  • Implement transfer learning using advanced CNN architectures
  • Use deep reinforcement learning techniques to solve optimization problems in PyTorch
  • Mix multiple models for a powerful ensemble model
  • Build image classifiers by implementing CNN architectures using PyTorch
  • Get up to speed with reinforcement learning, GANs, LSTMs, and RNNs with real-world examples

Who this book is for

This book is for data scientists and machine learning engineers looking to work with deep learning algorithms using PyTorch 1.x. You will also find this book useful if you want to migrate to PyTorch 1.x. Working knowledge of Python programming and some understanding of machine learning will be helpful.

Table of Contents

  1. Title Page
  2. Copyright and Credits
    1. Deep Learning with PyTorch 1.x Second Edition
  3. About Packt
    1. Why subscribe?
  4. Contributors
    1. About the authors
    2. About the reviewers
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  6. Section 1: Building Blocks of Deep Learning with PyTorch 1.x
  7. Getting Started with Deep Learning Using PyTorch
    1. Exploring artificial intelligence
      1. The history of AI
    2. Machine learning in the real world 
      1. So, why DL?
    3. Applications of deep learning
      1. Automatic translation of text from images
      2. Object detection in self-driving cars
    4. Deep learning frameworks
      1. Why PyTorch?
      2. What's new in PyTorch v1.x?
      3. CPU versus GPU
      4. What is CUDA?
      5. Which GPUs should we use?
      6. What should you do if you don't have a GPU?
    5. Setting up PyTorch v1.x
      1. Installing PyTorch
    6. Summary
  8. Building Blocks of Neural Networks
    1. What is a neural network?
      1.  Understanding the structure of neural networks
    2. Building a neural network in PyTorch
      1. PyTorch sequential neural network
      2. Building a PyTorch neural network using nn.Module
    3. Understanding PyTorch Tensors
      1. Understanding Tensor shapes and reshaping Tensors
    4. Understanding tensor operations
      1. Understanding Tensor types in PyTorch
      2. Importing our dataset as a PyTorch Tensor
      3. Training neural networks in PyTorch
    5. Summary 
  9. Section 2: Going Advanced with Deep Learning
  10. Diving Deep into Neural Networks
    1. Diving into the building blocks of neural networks
      1. Layers – the fundamental blocks of neural networks
    2. Non-linear activations
      1. Sigmoid
      2. Tanh
      3. ReLU
      4. Leaky ReLU
    3. PyTorch non-linear activations
      1. The PyTorch way of building deep learning algorithms
      2. Model architecture for different machine learning problems
      3. Loss functions
      4. Optimizing network architecture
    4. Image classification using deep learning
      1. Loading data into PyTorch tensors
      2. Loading PyTorch tensors as batches
      3. Building the network architecture
      4. Training the model
    5. Summary
  11. Deep Learning for Computer Vision
    1. Introduction to neural networks
      1. MNIST – getting data
    2. Building a CNN model from scratch
      1. Conv2d
      2. Pooling
      3. Nonlinear activation – ReLU
      4. View
        1. Linear layer
      5. Training the model
      6. Classifying dogs and cats – CNN from scratch
      7. Classifying dogs and cats using transfer learning
    3. Creating and exploring a VGG16 model
      1. Freezing the layers
      2. Fine-tuning VGG16
      3. Training the VGG16 model
    4. Calculating pre-convoluted features
    5. Understanding what a CNN model learns
      1. Visualizing outputs from intermediate layers
    6. Visualizing the weights of the CNN layer
    7. Summary
  12. Natural Language Processing with Sequence Data
    1. Working with text data
      1. Tokenization
        1. Converting text into characters
        2. Converting text into words
      2. N-gram representation
      3. Vectorization
        1. One-hot encoding
        2. Word embedding
    2. Training word embedding by building a sentiment classifier
      1. Downloading IMDb data and performing text tokenization
        1. Tokenizing with torchtext.data
        2. Tokenizing with torchtext.datasets
      2. Building vocabulary
      3. Generating batches of vectors
      4. Creating a network model with embedding
      5. Training the model
    3. Using pretrained word embeddings
      1. Downloading the embeddings
      2. Loading the embeddings in the model
      3. Freezing the embedding layer weights
    4. Recursive neural networks
      1. Understanding how RNN works with an example
    5. Solving text classification problem using LSTM
      1. Long-term dependency
      2. LSTM networks
        1. Preparing the data
        2. Creating batches
        3. Creating the network
        4. Training the model
    6. Convolutional network on sequence data
      1. Understanding one-dimensional convolution for sequence data
        1. Creating the network
        2. Training the model
    7. Language modeling
      1. Pretrained models
      2. Embeddings from language models
      3. Bidirectional Encoder Representations from Transformers
      4. Generative Pretrained Transformer 2
      5. PyTorch implementations
        1. GPT-2 playground
    8. Summary
  13. Section 3: Understanding Modern Architectures in Deep Learning
  14. Implementing Autoencoders
    1. Applications of autoencoders
      1. Bottleneck and loss functions
        1. Coded example – standard autoencoder
      2. Convolutional autoencoders
        1. Coded example – convolutional autoencoder
      3. Denoising autoencoders
    2. Variational autoencoders
      1. Training VAEs
        1. Coded example – VAE
    3. Restricted Boltzmann machines
      1. Training RBMs
      2. Theoretical example – RBM recommender system
        1. Coded example – RBM recommender system
    4. DBN architecture
      1. Fine-tuning
    5. Summary
    6. Further reading
  15. Working with Generative Adversarial Networks
    1. Neural style transfer 
      1. Loading the data
      2. Creating the VGG model
      3. Content loss
      4. Style loss
      5. Extracting the losses 
      6. Creating a loss function for each layer
      7. Creating the optimizer
      8. Training the model
    2. Introducing GANs
    3. DCGAN
      1. Defining the generator network
        1. Transposed convolutions
        2. Batch normalization
        3. Generator
      2. Defining the discriminator network
      3. Defining loss and optimizer
      4. Training the discriminator
        1. Training the discriminator with real images
        2. Training the discriminator with fake images
      5. Training the generator network 
      6. Training the complete network
      7. Inspecting the generated images
    4. Summary
  16. Transfer Learning with Modern Network Architectures
    1. Modern network architectures
      1. ResNet 
        1. Creating PyTorch datasets
        2. Creating loaders for training and validation
        3. Creating a ResNet model
        4. Extracting convolutional features
        5. Creating a custom PyTorch dataset class for the pre-convoluted features and loader
        6. Creating a simple linear model
        7. Training and validating the model
      2. Inception
        1. The Inception architecture
        2. Creating an Inception model
        3. Extracting convolutional features using register_forward_hook
        4. Creating a new dataset for the convoluted features
        5. Creating a fully connected model
        6. Training and validating the model
    2. Densely connected convolutional networks – DenseNet
      1. The _DenseBlock object
      2. The _DenseLayer object
        1. Creating a DenseNet model
        2. Extracting DenseNet features
        3. Creating a dataset and loaders
        4. Creating a fully connected model and training it
    3. Model ensembling
      1. Creating models
      2. Extracting the image features
      3. Creating a custom dataset, along with data loaders
      4. Creating an ensembling model
      5. Training and validating the model
    4. Encoder-decoder architecture 
      1. Encoder 
      2. Decoder
      3. Encoder-decoder with attention 
    5. Summary
  17. Deep Reinforcement Learning
    1. Introduction to RL
      1. Model-based RL
      2. Model-free RL
      3. Comparing on-policy and off-policy
    2. Q-learning
      1. Value methods
      2. Value iteration
      3. Coded example – value iteration
    3. Policy methods
      1. Policy iteration
        1. Coded example – policy iteration
      2. Value iteration versus policy iteration
      3. Policy gradient algorithm
        1. Coded example – policy gradient algorithm
    4. Deep Q-networks
      1. DQN loss function
      2. Experience replay
      3. Coded example – DQN
      4. Double deep Q-learning
    5. Actor-critic methods
      1. Coded example – actor-critic model
      2. Asynchronous actor-critic algorithm
    6. Practical applications
    7. Summary
    8. Further reading
  18. Whats Next?
    1. What's next?
      1. Overview of the book
      2. Reading and implementing research papers
    2. Interesting ideas to explore
      1. Object detection
      2. Image segmentation
      3. OpenNMT in PyTorch
      4. Allen NLP
      5. fast.ai – making neural nets uncool again
      6. Open neural network exchange
      7. How to keep yourself updated
    3. Summary
  19. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think
52.15.59.163