Neural Network related algorithms have existed for many decades. The first computational model was described by Warren McCulloch and Walter Pitts in 1943 in the Bulletin of Mathematical Biophysics.

Various man-made objects in the physical world, such as aeroplanes, have drawn inspiration from nature. A neural network is in essence a representation of the phenomenon of data exchange between the axons and dendrons (also known as dendrites) of neurons in the human nervous system. Just as data passes between one neuron to multiple other neurons to make complex decisions, an artificial neural network in similar ways creates a network of neurons that receive input from other neurons.

At a high level, an artificial neural network consists of 4 main components:

• Input Layer
• Hidden Layer(s)
• Output Layer
• Nodes and Weights

This is depicted in the following image:

Each node in the diagram produces an output based on the input from the preceding layer. The output is produced using an activation function. There are various types of activation functions and the output produced depends on the type of function used. Examples include binary step (0 or 1), tanh (between -1 and +1), sigmoid, and others.

The following diagram illustrates the concept:

The values x1 and x2 are the inputs, w1 and w2 represent the weights, and the node represents the point at which the inputs and their weights are evaluated and a specific output is produced by the activation function. The output f can thus be represented by:

Here, f represents the activation function, and b represents the bias term. The bias term is independent of the weights and the input values and allows the user to shift the output to achieve a better model performance.

Neural networks with multiple hidden layers (generally 2 or more) are computationally intensive, and in recent days, neural networks with multiple hidden layers, also known as deep neural networks or more generally deep learning, have become immensely popular.

A lot of the developments in the industry, driven by machine learning and artificial intelligence, have been the direct result of the implementation of such multi-layer neural networks.

In R, the package nnet provides a readily usable interface to neural networks. Although in practice, neural networks generally require sophisticated hardware, GPU cards, and so on for illustration purposes, we have leveraged the nnet package to run the earlier classification exercise on the PimaIndiansDiabetes dataset. In the example, we will leverage caret in order to execute the nnet model:

library(mlbench) 
library(caret) 
set.seed(123) 
 
 
data("PimaIndiansDiabetes") 
diab<- PimaIndiansDiabetes 
 
train_ind<- createDataPartition(diab$diabetes,p=0.8,list=FALSE,times=1) 
 
training_diab<- diab[train_ind,] 
test_diab<- diab[-train_ind,] 
 
nnet_grid<- expand.grid(.decay = c(0.5,0.1), .size = c(3,5,7)) 
 
nnet_model<- train(diabetes ~ ., data = training_diab, method = "nnet", metric = "Accuracy", maxit = 500, tuneGrid = nnet_grid) 

# Generating predictions using the neural network model
nnet_predicted <- predict(nnet_model, test_diab)

> plot (nnet_model)

# Confusion Matrix for the Neural Network model

confusionMatrix(nnet_predicted,test_diab$diabetes)

Confusion Matrix and Statistics 
 
          Reference 
Prediction negpos 
neg  86  22 
pos  14  31 
 
Accuracy : 0.7647           
                 95% CI : (0.6894, 0.8294) 
    No Information Rate : 0.6536           
    P-Value [Acc> NIR] : 0.001988         
 
Kappa : 0.4613           
Mcnemar's Test P-Value : 0.243345         
 
Sensitivity : 0.8600           
Specificity : 0.5849           
PosPredValue : 0.7963           
NegPredValue : 0.6889           
Prevalence : 0.6536           
         Detection Rate : 0.5621           
   Detection Prevalence : 0.7059           
      Balanced Accuracy : 0.7225           
 
       'Positive' Class :neg