After the discriminator has updated it's weights, we will train both the discriminator and generator together as a single model. When doing so, we will make the discriminator's weights non-trainable, freezing them in place but still allowing the discriminator to reverse propagate a gradient to the generator so that the generator can update it's weights.

For this step in the training process, we will use a noise vector as input, which will cause an image to be generated by the generator. The discriminator will be shown that image and asked to predict if the image is real or not. The following diagram illustrates this process:

The discriminator will come up with some prediction, which we can call . The loss function for this stack will be binary cross-entropy and we will pass the loss function a label of 1, which we can consider y. As you likely recall from earlier in the book, the loss between y and  is converted into a gradient that is passed back through the discriminator to the generator. This will update the generator weights, allowing it to benefit from the discriminator's knowledge of the problem space so that it can learn to create a more realistic generated image.

These two steps are then repeated over and over again, hopefully until the generator is able to create data that resembles the data in the training set to a point where the discriminator can no longer tell the two datasets apart and it becomes a guessing game for the discriminator. At this same point, the generator will no longer be able to improve. When we've found this Nash equilibrium, the network is trained.