Preface
Deep Learning (DL) is what humanizes machines. Deep Learning makes it possible for machines to see (through vision models), to listen (through voice devices like Alexa) to talk (through chatbots), to write (through generative models like auto-complete or Q&A) and even be an artist by trying to paint (through style transfer models).
PyTorch Lightning lets researchers build their own DL models quickly & easily without having to worry about the complexities. This book will help you maximize productivity for DL projects while ensuring full flexibility from model formulation to implementation.
The book provides a hands-on approach for implementing PyTorch Lightning DL models and associated methodologies that will have you up and running and productive in no time. You'll learn how to configure PyTorch Lightning on a cloud platform, understand the architectural components, and explore how they are configured to build various industry solutions. Next, you'll build a neural network architecture and deploy an application from scratch and see how you can expand it based on your specific needs, beyond what the framework can provide. The book also demonstrates how to implement capabilities to build and train various models like Convolutional Neural Nets (CNN), Natural Language Processing (NLP), Time Series, Self-Supervised Learning, Semi-Supervised Learning, Generative Adversarial Network (GAN) using PyTorch Lightning.
Who this book is for
If you have always been curious about DL but don't know where to start or feel intimidated by the complexities of large neural networks, then this book is ideal for you! It will make DL feel like a cakewalk!
This Deep Learning with PyTorch Lightning book is primarily for citizen data scientists who are beginning their DL journey and need a practical guide to ace their game! It is also beneficial to expert data scientists making the transition from other frameworks to PyTorch Lightning. This book will also appeal to DL researchers who are just getting started with coding for DL models using PyTorch Lightning by serving as a ready reckoner.
Working knowledge of Python programming and an intermediate level understanding of statistics and DL fundamentals is expected.
What this book covers
Chapter 1, PyTorch Lightning Adventure, will start with a brief history of Deep Learning and why PyTorch is the most preferred framework of the community today. We will also see what PyTorch Lightning is, how it is built, and how it differs from PyTorch. We will cover the module structure of PyTorch Lightning and how it makes research more feasible with users putting less effort into engineering and more into modelling.
Chapter 2, Getting off the Ground with the First Deep Learning Model, focuses on how to get started with building models using PyTorch Lightning. As examples, we will build multiple models, ranging from a very simple Multilayer Perceptron (MLP) to a real-life image recognition model using CNN.
Chapter 3, Transfer Learning Using Pre-Trained Models, mainly focuses on how to customize the models built using pre-trained architecture to achieve great results without large training budgets or time for different datasets. We will walk you through the steps of customizing models using pre-trained models for images and NLP.
Chapter 4, Ready-to-Cook Models from Lightning Flash , mainly focuses on PyTorch Lightning Flash – a state-of-the-art (SOTA) model architecture library. It includes most of the common algorithms or frameworks out of the box, thereby improving the productivity of data scientists by a huge margin for benchmarking & experimentation. This chapter will share various Flash models for video (video classification) and audio (automatic speech recognition) tasks.
Chapter 5, Time Series Models, mainly focuses on the working of time series models, with PyTorch implementation, along with step-by-step working and in-detail examples from basic to advanced time series techniques, such as (Recurrent Neural Networks) RNN and Long Short Term Memory (LSTM) models, along with real-world use cases.
Chapter 6, Deep Generative Models, focuses on the step-by-step, in-detail working and implementation of generative types of DL models, such as Generative Adversarial Networks (GANs), where it will be used to generate new non-existent images.
Chapter 7, Semi-Supervised Learning, mainly focuses on how semi-supervised models work, and how they can be implemented with PyTorch Lightning. We will also cover, in detail, working examples and implementation from basic to advanced semi-supervised models using PyTorch Lightning to handle label propagation and image caption generation using a combination of CNNs and RNNs.
Chapter 8, Self-Supervised Learning, mainly focuses on a new area of Self-Supervised Learning which can work on unlabeled data and explain how self-supervised models can be implemented with PyTorch Lightning. We will also cover working examples of contrastive learning and some techniques like SimCLR architecture.
Chapter 9, Deploying and Scoring Models, will cover, in detail, techniques and ways to deploy a DL model natively as well as in inter-operable formats such as ONNX. It will also cover, in detail, techniques and ways to perform model scoring on massive volumes of data.
Chapter 10, Scaling and Managing Training, will take a nuanced view of the challenges of training a model at scale and managing the training. It will describe some of the common pitfalls and tips and tricks on how to avoid them. It will also describe how to set up your experiments, how to make model training resilient and how to use the hardware to improve training efficiency, among other things.
To get the most out of this book
Getting started with PyTorch Lightning is very easy. You can use the Anaconda distribution to set up your environment locally or use a cloud option such as Google Colab, AWS, Azure, or IBM Watson Studio to get started. (It is recommended that you use a cloud environment with GPU to run some of the more complex models.)
Deep Learning Models in this book are trained using color images. Please also use digital version which has all the color images; to better understand the results.
PyTorch Lightning can be installed using pip in your Jupyter Notebook environment:
pip install pytorch-lightning
In addition to importing PyTorch Lightning (the first import statement can be seen as follows), the following import block shows statements that are usually part of the code:
import pytorch_lightning as pl
import torch
from torch import nn
import torch.nn.functional as F
from torchvision import transforms
The import packages and their versions change for each chapter, so please ensure that you are importing correct packages as mentioned on the Technical Requirements sections of the book.
The torch package is used for defining tensors and performing mathematical operations on the tensors. The torch.nn package is used for constructing neural networks, which is what nn stands for. torch.nn.functional contains functions including activation and loss functions, whereas torchvision.transforms is a separate library that provides common image transformations.
If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book's GitHub repository (a link is available in the next section). Please substitute correct installation & package versions as mentioned in the Technical Requirements sections before running GitHub files. Doing so will help you avoid any potential errors related to the copying and pasting of code.
Download the example code files
You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Deep-Learning-with-PyTorch-Lightning. If there's an update to the code, it will be updated in the GitHub repository.
We also have other code bundles from our rich catalogue of books and videos available at https://github.com/PacktPublishing/. Check them out!
Download the colour images
We also provide a PDF file that has colour images of the screenshots and diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781800561618_ColorImages.pdf.
Conventions used
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "You can use the gpus argument passed to Trainer to specify the number of GPUs."
A block of code is set as follows:
import pytorch_lightning as pl
...
# use only 10% of the training data for each epoch
trainer = pl.Trainer(limit_train_batches=0.1)
# use only 10 batches per epoch
trainer = pl.Trainer(limit_train_batches=10)
Any command-line input or output is written as follows:
pip install pytorch-lightning
Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: "For example, in Google Colab, you can Change Runtime Type to set Runtime shape to High-RAM instead of Standard so that you can increase the value of the num_workers argument to DataLoader "
Tips or Important Notes
Appear like this.
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Share Your Thoughts
Once you've read Deep Learning with PyTorch Lightning, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.
Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.