Machine learning has been instrumental in shaping the scope of technology since its inception. ML has played an important role in the development of things such as autonomous vehicles and robotics. Deep reinforcement learning is that field of learning where agents learn with help of rewards—a thought which has been derived from nature. Through this book, the author tries to present the diversity of reinforcement learning algorithms in game development as well as in scientific research. Unity, the cross-platform engine that is used in a plethora of tasks, from visual effects and cinematography to machine learning and high performance graphics, is the primary tool that is used in this book. With the power of the Unity ML Agents Toolkit, the deep reinforcement learning framework built by Unity, the author tries to show the vast possibilities of this learning paradigm.

The book starts with an introduction to state-based reinforcement learning, from Markov processes to Bellman equations and Q-learning, which sets the ground for the successive sections. A plethora of diverse pathfinding algorithms, from Dijkstra to sophisticated variants of A* star, have been provided along with simulations in Unity. The book also covers how navigation meshes work for automated pathfinding in Unity. An introduction to the ML Agents Toolkit, from standard process for installation to training an AI agent with deep reinforcement learning algorithm (proximal policy operation [PPO]) is provided as a starter. Along the course of this book, there is an extensive usage of the Tensorflow framework along with OpenAI Gym environments for proper visualizations of complex deep reinforcement learning algorithms in terms of simulations, robotics, and autonomous agents. Successive sections of the book involve an in-depth study of the variety of on- and off-policy algorithms, ranging from discrete SARSA/Q-learning to actor critic variants, deep Q-network variants, PPO, and their implementations using the Keras Tensorflow framework on Gym. These sections are instrumental in understanding how different simulations such as the famous Puppo (Unity Berlin), Tiny agents, and other ML Agents samples from Unity are created and built. Sections with detailed descriptions about how to build simulations in Unity using the C# software development kit for ML Agents and training them using soft actor critic (SAC), PPO, or behavioral cloning algorithms such as GAIL are provided.

The latter part of this book provides an insight into curriculum learning and adversarial networks with an analysis of how AI agents are trained in games such as FIFA. In all these sections, a detailed description of the variants of neural networks—MLP, convolution networks, recurrent networks along with long short-term memory and GRU and their implementations and performance are provided. This is especially helpful as they are used extensively during building the deep learning algorithms. The importance of convolution networks for image sampling in Atari-based 2D games such as Pong has been provided. The knowledge of computer vision and deep reinforcement learning is combined to produce autonomous vehicles and driverless cars, which is also provided as an example template (game) for the readers to build upon.

Finally, this book also contains an in-depth review of the Obstacle Tower Challenge, which was organized by Unity Technologies to challenge state-of-the-art deep reinforcement learning algorithms. Sections on certain evolutionary algorithms along with the Google Dopamine framework has been provided for understanding the vast field of reinforcement learning. Through this book, the author hopes to infuse enthusiasm and foster research among the readers in the field of deep reinforcement learning.