The generator network in our dummy GAN is a simple feed-forward neural network with five layers: an input layer, three hidden layers, and an output layer. Let's take a closer look at the configuration of the generator (dummy) network:
|
Layer # |
Layer name |
Configuration |
|
1 |
Input layer |
input_shape=(batch_size, 100), output_shape=(batch_size, 100) |
|
2 |
Dense layer |
neurons=500, input_shape=(batch_size, 100), output_shape=(batch_size, 500) |
|
3 |
Dense layer |
neurons=500, input_shape=(batch_size, 500), output_shape=(batch_size, 500) |
|
4 |
Dense layer |
neurons=784, input_shape=(batch_size, 500), output_shape=(batch_size, 784) |
|
5 |
Reshape layer |
input_shape=(batch_size, 784), output_shape=(batch_size, 28, 28) |
The preceding table shows the configurations of the hidden layers, and also the input and output layers in the network.
The following diagram shows the flow of tensors and the input and output shapes of the tensors for each layer in the generator network:
Let's discuss how this feed-forward neural network processes information during forward propagation of the data:
- The input layer takes a 100-dimensional vector sampled from a Gaussian (normal) distribution and passes the tensor to the first hidden layer without any modifications.
- The three hidden layers are dense layers with 500, 500, and 784 units, respectively. The first hidden layer (a dense layer) converts a tensor of a shape of (batch_size, 100) to a tensor of a shape of (batch_size, 500).
- The second dense layer generates a tensor of a shape of (batch_size, 500).
- The third hidden layer generates a tensor of a shape of (batch_size, 784).
- In the last output layer, this tensor is reshaped from a shape of (batch_size, 784) to a shape of (batch_size, 28, 28). This means that our network will generate a batch of images, where one image will have a shape of (28, 28).