A trained convolutional layer is made up of many feature detectors, called filters, that slide over an input image as a moving window. We will talk about what's inside a filter in a moment, but for now it can be a black box. Imagine a single filter that has already been trained. Maybe that filter has been trained to detect edges in images, which you might think of as transitions between dark and light. As it passes over the image, its output represents the presence and location of the feature it detects, which can be useful for a second layer of filters. Extending our thought experiment slightly further, now imagine a single filter, in a second convolutional layer, that has also already been trained. Perhaps this new layer has learned to detect right angles, where two edges that have been found by the previous layer are present. On and on we go; as we add layers, more intricate features can be learned. This concept of feature hierarchies is central to convolutional neural networks. The following image from Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks by Honglak Lee and others (2011) illustrates the idea of feature hierarchies extremely well:

This is a very powerful technique and it possesses several advantages over the flatten and classify method of deep learning we've previously used on MNIST. We will talk about those shortly, but first let's look deeper inside the filters.