Way back in Chapter 7, Training a CNN From Scratch, we used convolutions to slide a window over regions of an image to learn complex visual features. This allowed us to learn important local visual features, regardless of where in the picture those features might have been, and then hierarchically learn more and more complex features as our network got deeper. We typically used a 3 x 3 or 5 x 5 filter on a 2D or 3D image. You may want to review Chapter 7, Training a CNN From Scratch, if you are feeling rusty on your understanding of convolution layers and how they work.

It turns out that we can use the same strategy on a sequence of words. Here, our 2D matrix is the output from an embedding layer. Each row represents a word, and all the elements in that row are its word vector. Continuing with the preceding example, we would have a 10 x 128 vector, where there are 10 words in a row, and each word is represented by a 128 element vector space. We most certainly can slide a filter over these words.

The convolutional filter size changes for NLP problems. When we're building networks to solve NLP problems, our filter will be as wide as the word vector. The height of the filter can vary, with typical ranges being between 2 and 5. A height of 5 would mean that we're sliding our filter across five words at a time.

It turns out that for many NLP problems, CNNs work very well and they're much faster than LSTMs. It's hard to give exact rules about when to use an RNN/LSTM and when to use a CNN. Generally, if your problem requires a state, or learning something from very far back in the sequence, you're likely to be better off with an LSTM. If your problem requires detecting a particular set of words that describe the text, or a semantic feeling for a document, then a CNN will likely solve your problem faster and possibly better.