In general, a neural network receives a single vector as input (such as our MNIST example in Chapter 3, Beyond Basic Neural Networks – Autoencoders and RBMs) and then goes through several hidden layers, before arriving at the end with our inference for the result. This is fine for images that aren't that big; when our images become larger, however, as they usually are in most real-life applications, we want to ensure that we aren't building immensely large hidden layers to process them correctly.

Of course, one of the convenient features that is present in our ideas with tensors is the fact that we don't actually have to feed a vector into the model; we can feed something a little more complicated and with more dimensions. Basically, what we want to do with a CNN is that we want to have neurons arranged in three dimensions: height, width, and depth—what we mean by depth here is the number of colors in our color system, in our case being red, green, and blue.

Instead of trying to connect every neuron in a layer together, we will try to reduce it so that it is more manageable and less likely to be overfitted for our sample size, as we won't be trying to train every single pixel of the input.