We've already seen that the full action space is quite large (in the order of  ). You may have played video games where you only used a joystick with four arrow buttons or the arrow keys on a keyboard to control the speed and heading (the direction in which the car is pointed) to drive, so why can't we ask the agent to control the car in a similar way here? Well, that is the idea behind discretizing the action space. Although we won't be able to have precise control over the car, we can make sure that the discretized space gives us good control in a simulation environment.

Let's begin by using the similar convention we used in the continuous action space case – where we used one floating point value to represent the throttle (acceleration) and brake (deceleration) actions, thereby using a two-dimensional bounded space internally. This means the action space, in this case, can be defined as follows:

action_space = gym.spaces.Discrete(NUM_DISCRETE_ACTIONS)

As you can see here, NUM_DISCRETE_ACTONS is equal to the number of different actions available, which we will define later on in this section.

We will then discretize the space using two-dimensional bounded space and exposing this as the discrete action space to the agent. To keep the number of possible actions to a minimum while still allowing control over the car, we use the following list of actions:

Let's now define the previous set of discrete actions in a DISCRETE_ACTIONS dictionary in our carla_env implementation script, shown as follows:

DISCRETE_ACTIONS = {
    0: [0.0, 0.0],    # Coast
    1: [0.0, -0.5],   # Turn Left 
    2: [0.0, 0.5],    # Turn Right
    3: [1.0, 0.0],    # Forward
    4: [-0.5, 0.0],   # Brake
    5: [1.0, -0.5],   # Bear Left & accelerate
    6: [1.0, 0.5],    # Bear Right & accelerate
    7: [-0.5, -0.5],  # Bear Left & decelerate
    8: [-0.5, 0.5],   # Bear Right & decelerate
}