When I began using artificial intelligence tools in quantitative financial research, I could not find a comprehensive introductory text focusing on financial applications. Neural network libraries like TensorFlow, PyTorch, and Caffe had made tremendous contributions in the rapid development, testing, and deployment of deep neural networks, but I found most applications restricted to computer science, computer vision, and robotics. Having to use reinforcement learning algorithms in finance served as another reminder of the paucity of texts in this field. Furthermore, I found myself referring to scholarly articles and papers for mathematical proofs of new reinforcement learning algorithms. This led me to write this book to provide a one-stop resource for Python programmers to learn the theory behind reinforcement learning, augmented with practical examples drawn from the field of finance.

In practical applications, reinforcement learning draws upon deep neural networks. To facilitate exposition of topics in reinforcement learning and for continuity, this book also provides an introduction to TensorFlow and covers neural network topics like convolutional neural networks (CNNs) and recurrent neural networks (RNNs).

Finally, this book also introduces readers to writing modular, reusable, and extensible reinforcement learning code. Having worked on developing trading strategies using reinforcement learning and publishing papers, I felt existing reinforcement learning libraries like TF-Agents are tightly coupled with the underlying implementation framework and do not express central concepts in reinforcement learning in a manner that is modular enough for someone conversant with concepts to pick up TF-Agent library usage or extend its algorithms for specific applications. The code samples covered in this book provide examples of how to write modular code for reinforcement learning.

Reinforcement learning is a rapidly growing area of artificial intelligence that involves an agent learning from past experience of rewards gained by taking specific actions in certain states. The agent seeks to learn a policy prescribing the optimum action in each state with the objective of maximizing expected discounted future rewards. It is an unsupervised learning technique where the agent learns the optimum policy by past interactions with the environment. Supervised learning, by contrast, seeks to learn the pattern of output corresponding to each state in training data. It attempts to train the model parameters in order to get a close correspondence between predicted and actual output for a given set of inputs. This book outlines the theory behind reinforcement learning and illustrates it with examples of implementations using TensorFlow. The examples demonstrate the theory and implementation details of the algorithms, supplemented with a discussion of corresponding APIs from TensorFlow and examples drawn from quantitative finance. It guides a reader familiar with Python programming from basic to advanced understanding of reinforcement learning algorithms, coupled with a comprehensive discussion on how to use state-of-the-art software libraries to implement advanced algorithms in reinforcement learning.

Most applications of reinforcement learning have focused on robotics or computer science tasks. By focusing on examples drawn from finance, this book illustrates a spectrum of financial applications that can benefit from reinforcement learning.