Restricted Boltzmann Machine (RBM) is a two-layered neural network--the first layer is called the visible layer and the second layer is called the hidden layer. They are called shallow neural networks because they are only two layers deep. They were first proposed in 1986 by Paul Smolensky (he called them Harmony Networks^[1]) and later by Geoffrey Hinton who in 2006 proposed Contrastive Divergence (CD) as a method to train them. All neurons in the visible layer are connected to all the neurons in the hidden layer, but there is a restriction--no neuron in the same layer can be connected. All neurons are binary in nature:

RBM can be used for dimensionality reduction, feature extraction, and collaborative filtering. The training in RBMs can be divided into three parts: forward pass, backward pass, and then compare.