Deep learning involves a huge amount of matrix multiplications, and Graphic Processing Units (GPUs) are a very important aspect when one begins to learn deep learning. Without a GPU, the experiment process may take a day or more. With a good GPU, we can quickly iterate over deep learning networks and large training datasets, and run multiple experiments in a short amount of time. With TensorFlow, we can work on a single GPU or even multiple GPUs with ease. However, most machine learning platform installations are very complicated once GPUs get involved.

In this chapter, we are going to discuss GPUs and focus on a step-by-step CUDA setup and a GPU-based TensorFlow installation. We will start by installing Nvidia drivers, the CUDA Toolkit, and the cuDNN library. Then, we will install GPU-enabled TensorFlow with pip. Finally, we show how to use Anaconda to simplify the installation process even further.