Let's consider the famous taxi problem, which is available in OpenAI Gym as a test environment (https://gym.openai.com/envs/Taxi-v2/). The taxi environment is as follows:

This is a 5 x 5 grid-based environment where there are certain locations labeled R, G, Y, and B. The green cursor is the taxi. The taxi has six possible actions:

• Move south (intuitively, move down)
• Move north (intuitively, move up)
• Move east (intuitively, move right)
• Move west (intuitively, move left)
• Pick up passenger
• Drop off passenger

The pickup and dropoff locations are chosen randomly and are shown in blue and violet, respectively. The aim of the taxi is to go to the pickup location wherever the taxi is located in the grid-space and then to the dropoff location.

In the next section, we will cover the Temporal Difference (TD) prediction.