In this chapter, we will cover the following recipes:
- Building a perceptron
- Building a single layer neural network
- Building a deep neural network
- Creating a vector quantizer
- Building a recurrent neural network for sequential data analysis
- Visualizing the characters in an optical character recognition database
- Building an optical character recognizer using neural networks
Our brain is really good at identifying and recognizing things. We want the machines to be able to do the same. A neural network is a framework that is modeled after the human brain to simulate our learning processes. Neural networks are designed to learn from data and recognize the underlying patterns. As with all learning algorithms, neural networks deal with numbers. Therefore, if we want to achieve any real world task involving images, text, sensors, and so on, we have to convert them into the numerical form before we feed them into a neural network. We can use a neural network for classification, clustering, generation, and many other related tasks.
A neural network consists of layers of neurons. These neurons are modeled after the biological neurons in the human brain. Each layer is basically a set of independent neurons that are connected to the neurons the adjacent layers. The input layer corresponds to the input data that we provide, and the output layer consists of the output that we desire. All the layers in between are called hidden layers. If we design a neural network with more hidden layers, then we give it more freedom to train itself with higher accuracy.
Let's say that we want the neural network to classify data, based on our needs. In order for a neural network to work accordingly, we need to provide labeled training data. The neural network will then train itself by optimizing the cost function. This cost function is the error between actual labels and the predicted labels from the neural network. We keep iterating until the error goes below a certain threshold.
What exactly are deep neural networks? Deep neural networks are neural networks that consist of many hidden layers. In general, this falls under the realm of deep learning. This is a field that is dedicated to the study of these neural networks, which are composed of multiple layers that are used across many verticals.
You can check out a tutorial on neural networks to learn more at http://pages.cs.wisc.edu/~bolo/shipyard/neural/local.html. We will use a library called NeuroLab throughout this chapter. Before you proceed, make sure that you install it. You can find the installation instructions at https://pythonhosted.org/neurolab/install.html. Let's go ahead and look at how to design and develop these neural networks.