Inspired by the working of neurons in the human brain, the concept of neural networks was proposed by Frank Rosenblatt in 1957. To understand the architecture fully, it is helpful to briefly look at the layered structure of neurons in the human brain. (Refer to the following diagram to get an idea of how the neurons in the human brain are chained together.)
In the human brain, dendrites act as sensors that detect a signal. The signal is then passed on to an axon, which is a long, slender projection of a nerve cell. The function of the axon is to transmit this signal to muscles, glands, and other neurons. As shown in the following diagram, the signal travels through interconnecting tissue called a synapse before being passed on to other neurons. Note that through this organic pipeline, the signal keeps traveling until it reaches the target muscle or gland, where it causes the required action. It typically takes seven to eight milliseconds for the signal to pass through the chain of neurons and reach its destination:
Inspired by this natural architectural masterpiece of signal processing, Frank Rosenblatt devised a technique that would mean digital information could be processed in layers to solve a complex mathematical problem. His initial attempt at designing a neural network was quite simple and looked similar to a linear regression model. This simple neural network did not have any hidden layers and was named a perceptron. The following diagram illustrates it:
Let's try to develop the mathematical representation of this perceptron. In the preceding diagram, the input signals are shown on the left-hand side. It is a weighted sum of inputs because each of the inputs (x1, x2..xn) gets multiplied by a corresponding weight (w1,w2… wn) and then summed up:
Note that it is a binary classifier because the final output from this perceptron is true or false depending on the output of the aggregator (shown as ∑ in the diagram). The aggregator will produce a true signal if it can detect a valid signal from at least one of the inputs.
Let's now look into how neural networks have evolved over time.