In this example, we're using layer sizes that are appropriate for generating a 28 x 28 grayscale image, which is exactly what we will be doing later in our MNIST example. The arithmetic of generators can be a little tricky if you haven't worked with one before, so we will cover that as we walk through each layer. The following figure shows the architecture:

The input to the generator is just a 100 x 1 vector of randomness that we will call a noise vector. GANs tend to work best when this noise vector is generated from a normal distribution.

The first layer of the network is dense and fully connected. It provides us with a way to set up the linear algebra so that we end up with the right shape of output. For each convolutional block, we will end up doubling our first and second axis (the rows and columns that will eventually become the height and width of the image), while the number of channels gradually shrinks to 1. We eventually need the height and width of the output to be 28. So, we will need to start with a 7 x 7 x 128 tensor, so that it can move to 14 x 14 and then eventually 28 x 28. To accomplish this, we will size the dense layer as 128 x 7 x 7 neurons or 6,272 units. This allows us to reshape the output of the dense layer to 7 x 7 x 128. If this seems a little less than obvious now, don't worry, it will make sense after you code it.

After the fully connected layer, things are more straightforward. We're using convolutional layers, just like we always have. However, this time we're using them backwards. We're no longer using max pooling to down sample. Instead we're up-sampling, using the convolutions to build up our network as we learn visual features, and eventually outputting a tensor of the appropriate shape.

Typically, the activation of the last layer in the generator is the hyperbolic tangent and the elements within the training image matrices are normalized to be between -1 and 1. This is one of the many GAN hacks that I'll mention throughout the chapter. Researchers have discovered several hacks that have been empirically observed to help build stable GANs, most of which can be found on this Git by Soumith Chintala, who also happens to be one of the authors of the original DCGAN paper at https://github.com/soumith/ganhacks. The world of deep learning research is most certainly a small one.