A feedforward neural network is an ANN that consists of several perceptrons, which are organized into layers, as shown in the following diagram: input layer, output layer, and one or more hidden layers. The hidden layers have nothing to do with the outside world, hence the name. Each layer perceptron, also known as a neuron, has direct connections to the perceptrons in the next layer, whereas connections between two neurons carry a weight thats similar to the perceptron weights. So, all the perceptrons in one layer are connected with the perceptrons in the next layer, and the information is fed forward to the next layer. This diagram shows a network with a four-unit Input layer, corresponding to the size of the  feature vector of length 4, a four-unit Hidden layer, and a two-unit Output layer, where each unit corresponds to one class value:

A feedforward neural network learns by finding the relationship between input and output values, which are fed into the network multiple times. The most popular approach to training multilayer networks is backpropagation. In backpropagation, the calculated output values are compared with the correct values in the same way as in the delta rule. The error is then fed back through the network by various techniques, adjusting the weights of each connection in order to reduce the value of the error. The error is calculated using the squared difference between the output value of the network and the original output value. The error indicates how far we are from the original output values. This process is repeated for a sufficiently large number of training cycles, until the error is under a certain threshold.

A feedforward neural network can have more than one hidden layer, where each additional hidden layer builds a new abstraction atop the preceding layers. This often leads to more accurate models; however, increasing the number of hidden layers leads to two known issues:

• Vanishing gradients problem: With more hidden layers, the training with backpropagation becomes less and less useful for passing information to the front layers, causing these layers to train very slowly
• Overfitting: The model fits the training data too well and performs poorly on real examples

Let's look at some other networks structures that address these issues.