Currently, the recurrent neural network (RNN) is one of the most well-known and practical approaches used to construct deep neural networks. They are designed to process time-series data. Typically, data of this nature is found in the following tasks:
- Natural language text processing, such as text analysis and automatic translation
- Automatic speech recognition
- Video processing, for predicting the next frame based on previous frames, and for recognizing emotions
- Image processing, for generating image descriptions
- Time series analysis, for predicting fluctuations in exchange rates or company stock prices
In recurrent networks, communications between elements form a directed sequence. Thanks to this, it becomes possible to process a time series of events or sequential spatial chains. Unlike multilayer perceptrons, recurrent networks can use their internal memory to process sequences of arbitrary lengths. At the time of writing, many different architectural solutions for recurrent networks (from simple to complex) have been proposed. Currently, the most widespread recurrent network architectures are long short-term memory (LSTM) and gated recurrent unit (GRU).
In this chapter, we will consider the main architectural features of recurrent networks and learn how recurrent networks are used to process natural language; describe algorithms for converting elements of a natural language into mathematical abstractions; and write a program to evaluate the emotional characteristics of the language to perform sentiment analysis on movie reviews.
Specifically, the following topics will be covered in this chapter:
- An overview of the RNN concept
- Training RNNs using the concept of backpropagation through time
- Exploring RNN architectures
- Understanding natural language processing with RNNs
- Sentiment analysis example with RNNs