In the VAE, our loss function is composed of two parts:

• Generative loss: This loss compares the model output with the model input. This can be the losses we used in the autoencoders, such as L2 loss.
• Latent loss: This loss compares the latent vector with a zero mean, unit variance Gaussian distribution. The loss we use here will be the KL divergence loss. This loss term penalizes the VAE if it starts to produce latent vectors that are not from the desired distribution.

The following screenshot shows the loss for the VAE and that it is a combination of the generative loss and a loss on the latent space: