Generative adversarial network (GAN) is a generative model consisting of two networks that are jointly trained, called generator and discriminator.

The dynamics between these two networks are like those between a forger and an investigator. The forger tries to produce faithful imitations of authentic works of art, while the investigator tries to distinguish the fakes from the originals. In this analogy, the forger represents the generator and the investigator represents the discriminator. The generator accepts input values ​​belonging to a fixed distribution and tries to produce images similar to those of the dataset. The discriminator tries to distinguish the data created by the generator from those belonging to the dataset. These two networks are jointly coached:

• The discriminator tries to return output = 1 if the input belongs to the dataset and returns 0 if its input was generated by the generator
• The generator instead tries to maximize the possibility that the discriminator will make mistakes

The generator acquires a random input noise and tries to create a sample of data, while the discriminator takes input from either real-world examples or the generator, as shown in the following diagram:

For simplicity, the two opposing networks are of the multilayer perceptron type; however, the same structure can be modeled with deep networks. For example, to generate new images, instead of sampling data from a complex distribution, the approach used in these networks is to start from values belonging to a simple distribution or from random values. Subsequently, they are mapped through a second distribution that will be learned during the training.

In such a system, training leads to constant competition between generator and discriminator. Under these conditions, the optimization process can be carried out independently on both sides. Naming G(z) the generator and D(x) the discriminator, the training of the model aims to maximize the probability of the discriminator to assign 1 to values coming from the training set, instead of 0 to those produced by the generator. On the other hand, we want to teach the generator to minimize the following quantity:

The training is then performed by applying the gradient descent technique to the following expression:

This method originates from game theory, in particular from the method called two-player minimax game. The algorithms of this type adopt the strategy of minimizing the maximum possible loss resulting from the choice of a player. It can happen that, in the training process, the discriminator is not able to classify examples generated by real ones.