Cover image for Mastering PyTorch

Mastering PyTorch

Release Date: 2021/02/01
ISBN: 9781789614381
Topic:
0%
23
Chapters
0-1
Hours read
0k
Total Words
    Start Reading Now    
       Add to Wishlist       
View table of contents

Master advanced techniques and algorithms for deep learning with PyTorch using real-world examples

Key Features

  • Understand how to use PyTorch 1.x to build advanced neural network models
  • Learn to perform a wide range of tasks by implementing deep learning algorithms and techniques
  • Gain expertise in domains such as computer vision, NLP, Deep RL, Explainable AI, and much more

Book Description

Deep learning is driving the AI revolution, and PyTorch is making it easier than ever before for anyone to build deep learning applications. This PyTorch book will help you uncover expert techniques to get the most out of your data and build complex neural network models.

The book starts with a quick overview of PyTorch and explores using convolutional neural network (CNN) architectures for image classification. You'll then work with recurrent neural network (RNN) architectures and transformers for sentiment analysis. As you advance, you'll apply deep learning across different domains, such as music, text, and image generation using generative models and explore the world of generative adversarial networks (GANs). You'll not only build and train your own deep reinforcement learning models in PyTorch but also deploy PyTorch models to production using expert tips and techniques. Finally, you'll get to grips with training large models efficiently in a distributed manner, searching neural architectures effectively with AutoML, and rapidly prototyping models using PyTorch and fast.ai.

By the end of this PyTorch book, you'll be able to perform complex deep learning tasks using PyTorch to build smart artificial intelligence models.

What you will learn

  • Implement text and music generating models using PyTorch
  • Build a deep Q-network (DQN) model in PyTorch
  • Export universal PyTorch models using Open Neural Network Exchange (ONNX)
  • Become well-versed with rapid prototyping using PyTorch with fast.ai
  • Perform neural architecture search effectively using AutoML
  • Easily interpret machine learning (ML) models written in PyTorch using Captum
  • Design ResNets, LSTMs, Transformers, and more using PyTorch
  • Find out how to use PyTorch for distributed training using the torch.distributed API

Who this book is for

This book is for data scientists, machine learning researchers, and deep learning practitioners looking to implement advanced deep learning paradigms using PyTorch 1.x. Working knowledge of deep learning with Python programming is required.

Table of Contents

  1. Mastering PyTorch
  2. Why subscribe?
  3. Foreword
  4. Contributors
  5. About the author
  6. About the reviewer
  7. Packt is searching for authors like you
  8. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Download the color images
    6. Conventions used
    7. Get in touch
    8. Reviews
  9. Section 1: PyTorch Overview
  10. Chapter 1: Overview of Deep Learning using PyTorch
    1. Technical requirements
    2. A refresher on deep learning
    3. Activation functions
    4. Optimization schedule
    5. Exploring the PyTorch library
    6. PyTorch modules
    7. Tensor modules
    8. Training a neural network using PyTorch
    9. Summary
  11. Chapter 2: Combining CNNs and LSTMs
    1. Technical requirements
    2. Building a neural network with CNNs and LSTMs
    3. Text encoding demo
    4. Building an image caption generator using PyTorch
    5. Downloading the image captioning datasets
    6. Preprocessing caption (text) data
    7. Preprocessing image data
    8. Defining the image captioning data loader
    9. Defining the CNN-LSTM model
    10. Training the CNN-LSTM model
    11. Generating image captions using the trained model
    12. Summary
  12. Section 2: Working with Advanced Neural Network Architectures
  13. Chapter 3: Deep CNN Architectures
    1. Technical requirements
    2. Why are CNNs so powerful?
    3. Evolution of CNN architectures
    4. Developing LeNet from scratch
    5. Using PyTorch to build LeNet
    6. Training LeNet
    7. Testing LeNet
    8. Fine-tuning the AlexNet model
    9. Using PyTorch to fine-tune AlexNet
    10. Running a pre-trained VGG model
    11. Exploring GoogLeNet and Inception v3
    12. Inception modules
    13. 1x1 convolutions
    14. Global average pooling
    15. Auxiliary classifiers
    16. Inception v3
    17. Discussing ResNet and DenseNet architectures
    18. DenseNet
    19. Understanding EfficientNets and the future of CNN architectures
    20. Summary
  14. Chapter 4: Deep Recurrent Model Architectures
    1. Technical requirements
    2. Exploring the evolution of recurrent networks
    3. Types of recurrent neural networks
    4. RNNs
    5. Bidirectional RNNs
    6. LSTMs
    7. Extended and bidirectional LSTMs
    8. Multi-dimensional RNNs
    9. Stacked LSTMs
    10. GRUs
    11. Grid LSTMs
    12. Gated orthogonal recurrent units
    13. Training RNNs for sentiment analysis
    14. Loading and preprocessing the text dataset
    15. Instantiating and training the model
    16. Building a bidirectional LSTM
    17. Loading and preprocessing text dataset
    18. Instantiating and training the LSTM model
    19. Discussing GRUs and attention-based models
    20. GRUs and PyTorch
    21. Attention-based models
    22. Summary
  15. Chapter 5: Hybrid Advanced Models
    1. Technical requirements
    2. Building a transformer model for language modeling
    3. Reviewing language modeling
    4. Understanding the transformer model architecture
    5. Developing a RandWireNN model from scratch
    6. Understanding RandWireNNs
    7. Developing RandWireNNs using PyTorch
    8. Summary
  16. Section 3: Generative Models and Deep Reinforcement Learning
  17. Chapter 6: Music and Text Generation with PyTorch
    1. Technical requirements
    2. Building a transformer-based text generator with PyTorch
    3. Training the transformer-based language model
    4. Saving and loading the language model
    5. Using the language model to generate text
    6. Using a pre-trained GPT-2 model as a text generator
    7. Out-of-the-box text generation with GPT-2
    8. Text generation strategies using PyTorch
    9. Generating MIDI music with LSTMs using PyTorch
    10. Loading the MIDI music data
    11. Defining the LSTM model and training routine
    12. Training and testing the music generation model
    13. Summary
  18. Chapter 7: Neural Style Transfer
    1. Technical requirements
    2. Understanding how to transfer style between images
    3. Implementing neural style transfer using PyTorch
    4. Loading the content and style images
    5. Loading and trimming the pre-trained VGG19 model
    6. Building the neural style transfer model
    7. Training the style transfer model
    8. Experimenting with the style transfer system
    9. Summary
  19. Chapter 8: Deep Convolutional GANs
    1. Technical requirements
    2. Defining the generator and discriminator networks
    3. Understanding the DCGAN generator and discriminator
    4. Training a DCGAN using PyTorch
    5. Defining the generator
    6. Defining the discriminator
    7. Loading the image dataset
    8. Training loops for DCGANs
    9. Using GANs for style transfer
    10. Understanding the pix2pix architecture
    11. Summary
  20. Chapter 9: Deep Reinforcement Learning
    1. Technical requirements
    2. Reviewing reinforcement learning concepts
    3. Types of reinforcement learning algorithms
    4. Discussing Q-learning
    5. Understanding deep Q-learning
    6. Using two separate DNNs
    7. Experience replay buffer
    8. Building a DQN model in PyTorch
    9. Initializing the main and target CNN models
    10. Defining the experience replay buffer
    11. Setting up the environment
    12. Defining the CNN optimization function
    13. Managing and running episodes
    14. Training the DQN model to learn Pong
    15. Summary
  21. Section 4: PyTorch in Production Systems
  22. Chapter 10: Operationalizing PyTorch Models into Production
    1. Technical requirements
    2. Model serving in PyTorch
    3. Creating a PyTorch model inference pipeline
    4. Building a basic model server
    5. Creating a model microservice
    6. Serving a PyTorch model using TorchServe
    7. Installing TorchServe
    8. Launching and using a TorchServe server
    9. Exporting universal PyTorch models using TorchScript and ONNX
    10. Understanding the utility of TorchScript
    11. Model tracing with TorchScript
    12. Model scripting with TorchScript
    13. Running a PyTorch model in C++
    14. Using ONNX to export PyTorch models
    15. Serving PyTorch models in the cloud
    16. Using PyTorch with AWS
    17. Serving PyTorch model on Google Cloud
    18. Serving PyTorch models with Azure
    19. Summary
    20. References
  23. Chapter 11: Distributed Training
    1. Technical requirements
    2. Distributed training with PyTorch
    3. Training the MNIST model in a regular fashion
    4. Training the MNIST model in a distributed fashion
    5. Distributed training on GPUs with CUDA
    6. Summary
  24. Chapter 12: PyTorch and AutoML
    1. Technical requirements
    2. Finding the best neural architectures with AutoML
    3. Using Auto-PyTorch for optimal MNIST model search
    4. Using Optuna for hyperparameter search
    5. Defining the model architecture and loading dataset
    6. Defining the model training routine and optimization schedule
    7. Running Optuna's hyperparameter search
    8. Summary
  25. Chapter 13: PyTorch and Explainable AI
    1. Technical requirements
    2. Model interpretability in PyTorch
    3. Training the handwritten digits classifier – a recap
    4. Visualizing the convolutional filters of the model
    5. Visualizing the feature maps of the model
    6. Using Captum to interpret models
    7. Setting up Captum
    8. Exploring Captum's interpretability tools
    9. Summary
  26. Chapter 14: Rapid Prototyping with PyTorch
    1. Technical requirements
    2. Using fast.ai to set up model training in a few minutes
    3. Setting up fast.ai and loading data
    4. Training a MNIST model using fast.ai
    5. Evaluating and interpreting the model using fast.ai
    6. Training models on any hardware using PyTorch Lightning
    7. Defining the model components in PyTorch Lightning
    8. Training and evaluating the model using PyTorch Lightning
    9. Summary
  27. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think
13.58.112.1