This chapter will cover a slightly different kind of model to what we have seen so far. All the models presented until now belong to a type of model called a discriminative model. Discriminative models aim to find the boundaries between different classes. They are interested in finding P(Y|X)—the probability of output Y given some input X. This is the natural probability distribution to work with for classification, as you usually want to find a label Y, given some input X.

However, there is another type of model called a generative model. Generative models are built to model the distributions of different classes. They are interested in finding P(Y, X)—the probability distribution of output Y and input X occurring together. In theory, if you can capture the probability distribution of classes in your data, you will know more about it, and you will be able to calculate P(Y|X) using Bayes rule.

Generative models belong to the category of unsupervised learning algorithms. Unsupervised means that we don't need to have labeled data.

In this chapter, some key topics we will learn about are as listed:

• Autoencoders
• Variational autoencoders
• Generative adversarial networks
• Implementing various generative models for generating handwritten digits