One of the main inputs for our system will be a source image or photo. We all know that a machine learning (ML) or deep learning model can't just use a raw image. We need some processing and also we need to extract relevant features from the image, which can then be used for tasks such as recognition and classification.

An image-feature extractor should essentially take in an input image, extract a rich hierarchical feature representation from it, and represent the output in the form of a fixed-length dense vector. We have already seen the power of DCNNs for handling computer vision tasks. Here, we will leverage the power of transfer learning by using the pretrained VGG-16 model as a feature extractor to extract bottleneck features from all our images. Just as a quick refresher, the following diagram shows the VGG-16 model:

For the purpose of feature extraction, we will remove the top part of the model, the softmax layer, and use the remaining layers for getting dense feature vectors from our input images. This is typically part of the encoding process and the output is fed into the decoder where the captions are generated.