Imagine for a moment that your data is not a discrete body of text or a carefully cleaned set of records from your organization's data warehouse. Perhaps you would like to train an agent to navigate an environment. How would you begin to solve this problem? None of the techniques that we have covered so far are suitable for such a task. We need to think about how we can train our model in quite a different way to make this problem tractable. Additionally, with use cases where the problem can be framed as an agent exploring and attaining a reward from an environment, from game playing to personalized news recommendations, Deep Q-Networks (DQNs) are useful tools in our arsenal of deep learning techniques.

Reinforcement learning (RL) has been described by Yann LeCun (who was instrumental in the development of Convolutional Neural Networks (CNNs) and, at the time of writing, the director of Facebook AI Research) as the cherry on the cake of machine learning methods. In this analogy, unsupervised learning is the cake and supervised learning is the icing. What's important for us to understand here is that RL only solves a very specific case of problems, despite offering the promise of model-free learning, where you simply offer some scalar reward as your model optimizes successfully toward the goal you have specified.

This chapter will offer a brief background on why this is, and how RL fits into the picture more generally. Specifically, we will cover the following topics:

• What is a DQN?
• Learning about the Q-learning algorithm
• Learning about how to train a DQN
• Building a DQN for solving mazes