The following is an Nvidia-Docker hierarchy that we need to understand before installing it:

What is Docker? According to the Docker website, the keyword for it's lightweight. Docker containers running on a single machine share that machine's operating system kernel; they start instantly and use less compute and RAM. Images are constructed from filesystem layers and share common files (source: https://www.docker.com/what-container).

Essentially, Docker allows us to create a lightweight Virtual Machine (VM) in a container where we can house all of our applications and guarantee that the environment is going to be the same every time we enter this container. NVIDIA-Docker goes one step further and provides the appropriate linkage for our Docker containers to be able to interact with a GPU. This is a critical piece for our development environment. Once we have NVIDIA-Docker set up, the rest of the environment is fairly straightforward to integrate.

The installation is easy. I'd encourage you to visit the website and ensure that the directions haven't changed since publication: https://github.com/NVIDIA/nvidia-docker. At this time, NVIDIA-Docker2 is the latest version of development. When you install the NVIDIA-Docker2 system with these steps, it should allow you to upgrade periodically with  sudo apt upgrade.

When going to the website, you should see a set of instructions similar to this one:

Ubuntu 14.04/16.04/18.04, Debian Jessie/Stretch
# If you have nvidia-docker 1.0 installed: we need to remove it and all existing GPU containers
docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1 docker ps -q -a -f volume={} | xargs -r docker rm -f
sudo apt-get purge -y nvidia-docker

# Add the package repositories
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | 
  sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | 
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update

# Install nvidia-docker2 and reload the Docker daemon configuration
sudo apt-get install -y nvidia-docker2
sudo pkill -SIGHUP dockerd

# Test nvidia-smi with the latest official CUDA image
docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi

Now, let's go over each of these commands in detail.