Another type of Boltzmann Machine is Deep Boltzmann machine (DBM). This is a neural network similar to RBM, but instead of having only one layer of hidden nodes, DBMs have many. Each layer of neurons is connected only to those adjacent (the one immediately preceding and immediately following); here also, the neurons of the same layer are not interconnected. This structure allows the emergence of particular statistics from each layer that can capture new data features. The following diagram shows a DBM model with one visible layer and two hidden layers:
As we can see, connections are only between units in neighboring layers. Like RBMs and DBMs contain only binary units.
The DBMs model assigns the following probability to a visible vector v:
In the previous formula, the terms are defined as follows:
- v is the visible vector
- θ = (W(1),W(2)) are the model parameters, representing visible-to-hidden and hidden-to-hidden symmetric interaction terms
- h(1) and h(2) are hidden stochastic binary variables
- Z(θ) is the partition function
DBMs are particularly useful in the case of the recognition of objects or words. This is due to the great ability to learn complex and abstract internal representations using little labeled input data, instead of exploiting a large amount of unlabeled input data. However, unlike deep convolutional neural networks, DBMs adopt the inference and training procedure in both directions to better detect representations of input structures.