Our thought process always has a sequence. We always understand things in an order. For example, if we watch a movie, we understand the next sequence by connecting it with the previous one. We retain the memory of the last sequence and get an understanding of the whole movie. We don't always go back to the first sequence in order to get it.

Can a neural network act like this? Traditional ones typically cannot operate in this manner and that is a major shortcoming. This is where recurrent neural networks make a difference. It comes with a loop that allows information to flow:

Here, a neural network takes an input as X_t and throws an output in the form of h_t . A recurrent neural network is made up of multiple copies of the same network that pass on the message to the successor.

If we were to go and unroll the preceding network, it would look like the following:

This chain-like nature reveals that recurrent neural networks are intimately related to sequences and lists. They are the natural architecture of neural networks to use for such data. Since the network has got an internal memory, RNNs are able to remember the input they received which, in turn, enables them to come up with accurate and precise results and predictions.

So far, we have been talking about sequential data. But we need to have a proper understanding of this term, sequential data. This form of data is an order data where there exists a relationship between data at time t and the data at time t-1. An example of that kind of data can be financial data, time-series data, video, and so on. RNNs allow us to operate over sequences of vectors. For example, look at the following image:

Each rectangle is represented as a vector, and arrows stand for functions. Input vectors are in red, output vectors are in blue, and green vectors hold the RNN's state:

• Vanilla mode of processing can be done without including RNN, from a fixed-sized input to output
• Sequencing the output in a proper format
• Sequencing the input 
• Sequencing the input and output (for example, machine translation: an RNN which reads a sentence in English and then outputs a sentence in some other language, like German).
• Syncing the sequenced input and output (for example, video classification where label each frame of the video)