The CartPole agent will use a fairly modest neural network that you should be able to train fairly quickly even without a GPU. We will start by looking at the model architecture as always. Then we will define the network's memory, exploration policy, and finally train the agent.