By now, you should understand the significance and meaning of image captioning. This task can be simply defined as writing and recording a free-flowing and natural text description for any image. It is usually used to describe various scenes or events in images. This is also popularly termed scene recognition. Let's look at the following example:

Looking at this scene, what could be a suitable caption or description? The following are all valid descriptions of the scene:

• A motocross rider is on a dirt hill
• A guy on a bicycle midair above a hill
• A dirt bike rider is moving fast down a dirt path
• A biker riding a black motorbike in midair

You can see that all of these captions are valid and are similar yet use different words to convey the same meaning. This is why generating image captions automatically is not an easy task.

In fact, the exact same thing is mentioned by a popular paper Show and Tell: A Neural Image Caption Generator, Vinyals and its co-authors, 2015 (https://arxiv.org/abs/1411.4555) describing image captioning, from which we draw our inspiration for building this system:

Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing.

For a human being, just glancing at a photo or image for a few seconds is enough to generate a natural language-based caption. However, enabling artificial intelligence (AI) to perform this task is extremely challenging given that the majority of computer vision problems have been focused on recognition and classification problems. Here are some of the major tasks with regard to core computer vision problems in terms of increasing complexity:

• Image classification and recognition: This involves a classic supervised learning problem where the main objective is to assign an image to a particular category based on several predefined class categories (often known as class labels). The popular ImageNet competition is one such task.
• Image annotation: A slightly more complex task, where we try to annotate an image with descriptions of various entities in the image. Typically, this involves categories or even natural language-based textual descriptions for specific sections or regions in the image.
• Image captioning or scene recognition: Another complex task where we try to describe an image with an accurate natural language-based textual description. This is the main area of focus for this chapter.

The task of image captioning is not a new thing. There have been several prior approaches of leveraging techniques such as stitching together text descriptions of individual entities from an image to form a description or even using template-based text-generation techniques. However, using deep learning is a more robust and efficient approach for this task.