In the last chapter, we trained an LSTM with a set of lags from a time series. Here our lags are really the words in a sequence. We will use these words to predict the sentiment of the reviewer. In order to get from a sequence of words to an input vector that considers the semantic value of those words, we can use an embedding layer.

Using the Keras functional API, the embedding layer is always the second layer in the network after the input layer. Let's look at how these two layers fit together:

input = Input(shape=(sequence_length,), name="Input")
embedding = Embedding(input_dim=vocab_size, output_dim=embedding_dim,
                      input_length=sequence_length, name="embedding")(input)

Our input layer needs to know the sequence length, which corresponds to the number of columns in the input matrix.

The embedding layer will use the input layer; however, it needs to know the overall corpus vocabulary size, the size of the vector space we're embedding those words into, and the sequence length.

We've defined a vocabulary of 20,000 words, the data has a sequence length of 2,494, and we've specified an embedding dimension of 100.

Putting this all together, the embedding layer will go from a 20,000 input one hot vector to a 2,494 x 100 2D matrix for each document yielding the vector space embedding for each word in the sequence. As the model learns, the embedding layer will learn along the way. Pretty cool, right?