In the last section, I said that a convolutional layer is a set of filters that act as feature detectors. Before we move too deep into that architecture, let's review the mathematics of what convolution actually is.
Let's start by manually convolving the following 4 x 4 matrix with a 3 x 3 matrix that we will call a filter. The first step in the convolution process is to take the element wise product of the filter and the first nine boxes of the 4 x 4 matrix:
Once we've carried this operation out, we will just slide the filter over one row and do the same thing. Finally, we will slide the filter down, and then over once again. The convolution process, once complete, will leave us with 2x2 matrix, as shown in the following figure:
Technically, this isn't a convolution, but a cross-correlation. We call it a convolution by convention and the difference for our purposes is really quite small.